MALIBA-AI/bambara-asr-leaderboard

bambara asr leardboard # Bambara ASR Leaderboard A public benchmark and leaderboard for evaluating Automatic Speech Recognition on Bambara (Bamanankan), a language spoken by over 14 million people in Mali and across West Africa. This repository contains the source code for the leaderboard application, deployed on Hugging Face Spaces with results persisted via GitHub. ## Benchmark All evaluations run against the **Bambara ASR Benchmark**: 1 hour of studio-recorded Malian constitutional text, transcribed and validated by linguists from Mali's DNENF-LN. | | | |:--|:--| | Language | Bambara (bm) | | Domain | Malian Constitution legal/formal register | | Duration | 1.075 hours, 518 segments | | Speaker | 1 adult male (main), 1 female (national anthem) | | Vocabulary | 1,198 unique words, ~75% absent from existing training corpora | | Acoustic conditions | Studio, single-channel, 99% of segments ≥ 15 dB SNR | | Code-switching | None | The benchmark is intentionally narrow: formal vocabulary, clean audio, no code-switching. It tests domain robustness and OOV handling under near-optimal conditions. Details in the paper. ## Scoring Transcriptions are normalized before scoring: lowercase, punctuation removed, whitespace collapsed. **Default ranking metric:** Combined Score = 0.5 × WER + 0.5 × CER The leaderboard UI lets you adjust WER/CER weights to match your use case. ## Submitting Results 1. Download the test audio from the benchmark dataset. 2. Run your model and generate transcriptions. 3. Format output as CSV with columns `id,text`, where `id` matches the dataset segment IDs. 4. Go to the leaderboard and upload via the **Submit New Results** tab. Scores are computed automatically on submission. ## Running Locally ```bash git clone github.com cd bambara-asr-leaderboard pip install -r requirements.txt python app.py ``` ## Current Top Results Check the leaderboard leaderboard. ## Contributing Contributions welcome new metrics, UI …