JudeTulel/whisper-kalenjin-lora

A weekend LoRA fine-tune of openai/whisper-large-v3-turbo on Kalenjin (~6M speakers in Kenya, not in Whisper's 99 languages). of GPU time, scripted WER 124% → 56%. Companion to tonykipkemboi.com # whisper-kalenjin-lora A weekend LoRA fine-tune of openai/whisper-large-v3-turbo on **Kalenjin** — a Nilotic language spoken by ~6 million people in Kenya, and not in the 99 languages Whisper was trained on. Total spend: ~$25 of GPU time on Modal. **Read the writeup:** Fine-tuning Whisper on Kalenjin: a $25 LoRA experiment. **Models on Hugging Face:** - `Tonykip/whisper-kalenjin-lora-v3-turbo` — LoRA adapter only (~50 MB) - `Tonykip/whisper-kalenjin-v3-turbo` — merged full model (~1.6 GB), drop-in replacement for the base --- ## Headline numbers All under the recommended decoding recipe (chunked + beam=5, normalized refs+preds). Source: `04-results/artifacts/canonical_metrics.json`. | Metric | Value | | --------------------------- | ---------- | | WER (overall, normalized) | **65.56%** | | CER (overall, normalized) | **21.10%** | | Mean coverage | 0.913 | | Kipsigis WER (n=156) | 65.51% | | Nandi WER (n=42) | 66.08% | | Dialect gap | −0.57 pts | For comparison: zero-shot base `whisper-large-v3-turbo` on Kalenjin scripted speech is **124% WER** (the model hallucinates Icelandic-looking text and falls into repetition loops). --- ## Repository layout ``` 00-overview.md — scope, day-by-day timeline 01-setup/ — compute & environment notes; dataset download script 02-dataset/ — dataset overview + download report 03-training/ — modal_train.py (the training script) 04-results/ — eval scripts, analysis, and metric artifacts audit_recompute.py — single-source-of-truth metrics recomputation normalize_unscripted_wer.py — strips [cs]/[pause] markup before scoring artifacts/ — JSON eval outputs, predictions, candidates *.md — analysis notes (transcripts, base64 bug, etc.) assets/charts/ — make_charts.py + the rendered …