ridwanbello/afrispeech-peft

Parameter-efficient fine-tuning of Whisper for African-accented English ASR using DoRA on AfriSpeech # AfriSpeech-PEFT: Parameter-Efficient Fine-tuning for African-Accented English ASR ## Overview We apply DoRA (Weight-Decomposed Low-Rank Adaptation) to fine-tune Whisper-medium and wav2vec2-xlsr-53 on AfriSpeech-200 (arxiv.org) and compare with the conventional full-finetuning approach Key findings: - Despite using only <5% of model parameters, Whisper finetuned with DoRA all-domain achieves 0.130 test WER vs 0.125 for full fine-tuning (100%) - DoRA reduces catastrophic forgetting by 61.7% on LibriSpeech (0.049 vs 0.128 for full fine-tuning) - DoRA generalises better cross-domain: general-trained DoRA achieves 0.246 on clinical test vs 0.266 for full fine-tuning - Full fine-tuning outperforms DoRA for wav2vec2-xlsr-53 on AfriSpeech, but DoRA still reduces forgetting (0.133 vs 0.158) ## Dataset AfriSpeech-200 — 200 hours of pan-African accented English speech from 2,463 speakers across 120 accents from 13 countries, covering general and clinical domains. ## Models The finetuned models are hosted on HuggingFace | Model | HuggingFace Hub | |---|---| | Whisper-medium full fine-tuned (general) | `robello2/whisper-medium-afrispeech-general` | | Whisper-medium full fine-tuned (clinical) | `robello2/whisper-medium-afrispeech-clinical` | | Whisper-medium full fine-tuned (all) | `robello2/whisper-medium-afrispeech-all` | | Whisper-medium DoRA (general) | `robello2/whisper-medium-dora-afrispeech-general` | | Whisper-medium DoRA (clinical) | `robello2/whisper-medium-dora-afrispeech-clinical` | | Whisper-medium DoRA (all) | `robello2/whisper-medium-dora-afrispeech-all` | | wav2vec2-xlsr-53 full fine-tuned (all) | `robello2/wav2vec2-xlsr-afrispeech-all` | | wav2vec2-xlsr-53 DoRA (general) | `robello2/wav2vec2-xlsr-dora-afrispeech-general` | | wav2vec2-xlsr-53 DoRA (clinical) | `robello2/wav2vec2-xlsr-dora-afrispeech-clinical` | | wav2vec2-xlsr-53 DoRA (all) | `robello2/wav2vec2-xlsr-dora-afrispeech-all` | ## Setup ```bash # Create environments pyt …