Olameta/llm-hallucination-african-education

Empirical study of hallucination patterns in DistilGPT-2 across globally represented vs. African educational questions, with attention visualization and alignment implications. # llm-hallucination-african-education Empirical bias audit of AI-driven educational prediction in low-resource African contexts — demonstrating how demographic-heavy models fail underrepresented learners. XGBoost, SHAP attribution, OULAD dataset. # Evaluating Hallucination Patterns in Small Language Models on African Educational Questions **Author:** Abdussomad Olayiwola **Affiliation:** LAUTECH, Nigeria | HLF 2025 Young Researcher **GitHub:** github.com **Contact:** olayiwolaabdussomad@gmail.com --- ## Research Question Do small open-source language models exhibit systematic reliability gaps when answering questions about African educational topics compared to globally represented content — and if so, what patterns characterize those failures? --- ## Motivation Large language models are increasingly deployed in educational settings worldwide. However, pre-training corpora are heavily skewed toward Western and English-language content. This raises a concrete AI safety concern: if models hallucinate more frequently on questions about underrepresented regions, then communities in those regions face systematically worse outcomes from AI-assisted learning tools. This study investigates whether this reliability gap exists empirically in DistilGPT-2 (82M parameters), treating it as a proxy for studying how training data distribution shapes model reliability across populations. This work connects to core AI safety concerns: - **Evaluation coverage:** Standard benchmarks may be blind to failures affecting specific populations - **Distributional harm:** Models that perform well on aggregate metrics can still fail specific communities - **Training data bias:** Reliability may track corpus representation rather than question difficulty --- ## Methodology ### Model - **DistilGPT-2** (82M parameters, HuggingFace) - A distilled version of GPT-2, chosen for accessibility and reproducibility on free compute (Google Colab CPU) - Prompted using a …