osinkolu/yecs-asr-benchmark

Finetune Omnilingual ASR on YECS; benchmark vs MMS/Whisper (Yoruba-English code-switch) # YECS ASR — Omnilingual finetune, benchmark & language-tag injection Finetune Meta **Omnilingual ASR** on the **Yoruba-English Code-Switching (YECS)** corpus (LyngualLabs), benchmark it against MMS and Whisper baselines, and study **inline language-tag injection** for joint transcription + per-word language ID. ## Result (held-out YECS test, 9,905 utts, unified normalization) | Model | WER | CER | |---|---|---| | **omni LLM (~1.63B, 5 epochs)** | **16.15%** | **6.30%** | | whisper-small-yoruba (reported) | 20.76% | — | | mms-300m-yoruba-english (reported) | 29.11% | — | Model: huggingface.co ## Language-tag injection (`tag-injection/`) Train the ASR to emit inline ` / ` language markers, e.g. ` first aid ni wọ́n ṣáré fún kí ó tó kú `, so one model transcribes **and** labels each word's language. Language-boundary prediction is learned as an auxiliary task at **zero extra inference cost**. Controlled A/B across **four architectures** × **two tag serializations** (segment spans vs. word-prefix), identical 5-epoch training, same 9,905-utt test, tags stripped before WER. WER tone-aware (plain → segment → word) and best-tag per-word LID: | Model | Loss / tokenizer | plain | segment | **word** | LID (seg → word) | |---|---|:---:|:---:|:---:|:---:| | Omni-CTC 300M | CTC / subword | 33.57 | 32.45 | **31.45** | 98.04 → **99.10** | | MMS-300m | CTC / **character** | **19.93** | 24.20 | 21.80 | 99.41 → **99.59** | | Whisper-small | seq2seq / subword | **16.73** | 16.81 | 17.18 | 99.54 → **99.55** | | Omni-LLM 1.63B | seq2seq / subword | **16.15** | 16.29 | 16.19 | 99.55 → **99.60** | **Findings:** (1) per-word LID is ~free everywhere (98–99.6%), and word-prefix gives the higher LID on all four models. (2) The best serialization is set by the *alignment mechanism*: **CTC** (frame-local) clearly prefers **word-level** — it more than halves the character-CTC penalty on MMS — while **seq2seq** (attention) is roughly indifferent f …