MaDi-12/wolof-language-detection-eval

# Wolof Language Detection — Manually Validated Evaluation This repository provides a comparative evaluation of language identification models (fastText, GlotLID, AfroLID) for Wolof detection. ## Motivation Reliable benchmarks for low-resource African languages remain scarce. This work constructs a manually validated evaluation set through native-speaker inspection and uses it to compare three LID systems in a Wolof-focused diagnostic benchmark. ## Models evaluated - fastText - GlotLID - AfroLID ## Methodology 1. Sampling from the output column of the MURI-IT test split 2. Manual re-annotation by a native Wolof speaker (247 instances, 235 confirmed Wolof, 11 non-Wolof, 1 mixed) 3. Binary evaluation: Wolof vs. non-Wolof 4. Metrics: Precision, Recall, F1-score, Accuracy, Confusion matrix ## Key findings - AfroLID achieves the best F1-score for Wolof (0.974) - GlotLID is conservative and misses many valid Wolof instances (recall: 0.677) - fastText fails to identify Wolof in an open multilingual setting (F1: 0.000) ## Data source This work uses the MURI-IT dataset for evaluation. The original dataset is not redistributed here. For access, see the official Hugging Face page: huggingface.co ## Reproducibility Run the notebook: ```bash pip install -r requirements.txt ```