Aymenec-212/darija-asr-comparison

# Darija-ASR · End-to-End vs Cascaded Architectures for Moroccan Darija Speech Recognition > A controlled comparison of **Qwen2-Audio-7B** (end-to-end audio-LLM) and **Whisper-Large-v3** (cascaded ASR) for Moroccan Darija (الدارجة المغربية), fine-tuned with LoRA under the **Marco-ASR** adaptive learning-rate framework. Same data, same splits, same seeds, same evaluation protocol — only the architecture changes. --- ## 🎯 Headline Result On **MoulSot** (in-distribution Darija ASR), **Whisper-Large-v3** beats **Qwen2-Audio-7B** at every training scale, by a margin that holds in both the code-switched and monolingual partitions, and on out-of-distribution audio (Casablanca-Morocco). The gap closes with more data but does not vanish. | Model | Params | 10h WER | 30h WER | 30h OOD WER | 30h GPU-hr | |----------------------------|---------|------------------|------------------|-------------------|------------| | Qwen2-Audio-7B (zero-shot) | 8.4B | 105.90% | — | 131.40% | — | | Whisper-LV3 (zero-shot) | 1.5B | 84.85% | — | 96.62% | — | | Qwen2-Audio-7B + LoRA | +9.4M | 57.76% | 47.58% | 63.98% | 8.89 | | **Whisper-LV3 + LoRA** | **+15M**| **40.14%** ⭐ | **39.23%** ⭐ | **58.92%** ⭐ | **3.41** | (95% bootstrap CIs in § Detailed results. Lower is better.) **Takeaways:** 1. Whisper-LV3 wins on every partition at every scale, with **~8 percentage points** absolute WER gap at 30h. 2. Whisper trains **~2.6× faster** at 30h despite identical effective batch and LoRA rank — direct consequence of being **5.6× smaller**. 3. The architectural mismatch is real: the LLM-based Qwen2-Audio starts much further from Darija (>100% WER zero-shot, MSA fallback + hallucinations) and never catches up within 30 hours of data. 4. Code-switching costs Whisper more than monolingual at e …