leangoogs-foundation/african-asr-eval

# african-asr-eval **Measuring what production speech recognition actually does to African languages on a phone line.** Every ASR vendor reports word error rate on clean 16 kHz studio audio. Almost every real deployment in Africa is an 8 kHz G.711 phone call. Nobody publishes the second number. This repository measures it, and gives away the instrument. **Live demo: The Telephony Test** — upload a clip, watch a phone line eat the accuracy. --- ## A result before you run anything | Language | Speakers | Whisper | Deepgram Nova | Google STT | |---|---|---|---|---| | Kiswahili | ~200M | yes (`sw`) | yes | yes | | Hausa | ~80M | yes (`ha`) | no | yes | | Yorùbá | ~45M | yes (`yo`) | no | yes | | **Igbo** | **~30M** | **NO CODE** | no | yes | | **Nigerian Pidgin** | **~120M** | **NO CODE** | no | no | Roughly 150 million people speak Igbo or Nigerian Pidgin. Whisper, the most widely deployed open speech model in the world, has no language token for either one. Not a poor score. No token. Production systems route Pidgin to the English decoder and accept the output. This benchmark records that as a documented choice rather than a silent fallback, because the fallback is the finding. *Corrections welcome via pull request, especially from vendors. This table is maintained by hand and support changes without notice.* --- ## What's here ``` tools/why_telephony_hurts.py Measured, dependency-free. Start here space/ The live Gradio demo paper1_tokenizer_fertility/ What African languages cost in tokens paper2_asr_benchmark/ The WER harness and gold protocol GOLD_PROTOCOL.md How the reference transcripts are made src/normalize.py Three normalisation conditions src/telephony.py 8 kHz + codec pipeline src/scoring.py Levenshtein by hand, plus jiwer src/systems.py Vendor adapters ``` Start with: ```bash python3 tools/why_telephony_hurts.py ``` It needs …