Endalk-Chala/ethiopia-ai-moderation-audit

# Auditing AI Moderation in Ethiopia's Multilingual Online Sphere — Replication Materials Replication data and code for **Chala, E. H. (2026). From Detection to Counterspeech: Auditing AI Moderation and Fact-Checking Practices in Ethiopia's Multilingual Online Sphere.** *Media and Communication*, 14, Article 12653. doi:10.17645/mac.12653 The study audits three publicly deployable hate-speech classifiers against 838 hand-annotated Amharic and Afan Oromo social-media posts (2020–2025). This repository holds the anonymized corpus, the classifier predictions, the annotation codebook and datasheet, the audit notebooks, and a script that regenerates the article's Table 1 in one command. **Version 1.0.0** --- ## What the audit found | Classifier | Language | Precision | Recall | F1 | Coverage | |---|---|---|---|---|---| | Perspective API (native) | All | — | — | — | 0/838 | | Perspective API (translation) | All | 1.00 | .10 | .19 | 687/838 | | AfriHate (AfroXLMR, fine-tuned) | Amharic | .91 | .64 | .75 | 470/470 | | AfriHate (AfroXLMR, fine-tuned) | Afan Oromo | .94 | .17 | .29 | 368/368 | | Amharic mBERT | Amharic | .86 | .56 | .68 | 470/470 | The most widely used generic classifier cannot read either language natively — every native-mode request is rejected. On human English translations it recovers about a tenth of the hate speech present. Locally oriented classifiers do substantially better on Amharic but collapse on Afan Oromo, for which no usable detection exists at any tier. Across every tool the dominant error is under-detection, not over-removal. Full table, per-language figures and error composition: `python reproduce.py`. ## Reproduce it ```bash pip install -r requirements.txt python reproduce.py ``` This verifies the corpus against every count and κ reported in the article — exiting non-zero on any mismatch — then regenerates Table 1 and the Section 5.1.2 error composition. Deterministic; no API key, no network, no GPU. To re-run the classifiers the …