SunbirdAI/kinyarwanda-whisper-eval

Evaluating Whisper performance wrt hours of labelled kinyarwanda audio # Kinyarwanda Whisper Evaluation Evaluating Whisper model performance on Kinyarwanda across different amounts of labeled audio. --- ## 📦 Install ```bash git clone github.com cd kinyarwanda-whisper-eval uv sync ``` Install SALT: ```bash git clone github.com uv pip install -r salt/requirements.txt ``` Set up environment: ```bash cp env_example .env ``` Fill in your `.env` with MLflow and Hugging Face credentials. --- ## 🚀 Usage ### Baseline evaluation (no training) ```bash uv run python train.py --config configs/baseline.yaml ``` ### Fine-tuning experiments ```bash uv run python train.py --config configs/train_1h.yaml uv run python train.py --config configs/train_10h.yaml uv run python train.py --config configs/train_50h.yaml uv run python train.py --config configs/train_500h.yaml ``` ### Evaluation ```bash uv run python eval.py --model_path --batch_size=8 ``` --- ## 📁 Training Configs | Config | Hours | Model ID on Hugging Face | | ------------------ | ------ | ----------------------------------- | | `baseline.yaml` | 0 | openai/whisper-large-v3 | | `train_1h.yaml` | 1 | akera/whisper-large-v3-kin-1h-v2 | | `train_50h.yaml` | 50 | akera/whisper-large-v3-kin-50h-v2 | | `train_100h.yaml` | 100 | akera/whisper-large-v3-kin-100h-v2 | | `train_150h.yaml` | 150 | akera/whisper-large-v3-kin-150h-v2 | | `train_200h.yaml` | 200 | akera/whisper-large-v3-kin-200h-v2 | | `train_500h.yaml` | 500 | akera/whisper-large-v3-kin-500h-v2 | | `train_1000h.yaml` | 1000 | akera/whisper-large-v3-kin-1000h-v2 | | `train_full.yaml` | \~1400 | akera/whisper-large-v3-kin-full | Explore the collection: 👉 huggingface.co --- ## 📊 Results Evaluation on `dev_test[:300]` subset: | Model | Hours | WER (%) | …