kh-ch-2000/Algerian-Arabic-Dialect-ASR

A comparative study and benchmark of Whisper Small, Wav2Vec2-XLSR-53, and MMS-1B fine-tuned on 800 minutes of Algerian Arabic Dialect (Darija), evaluating WER and CER # 🇩🇿 Algerian Arabic Dialect ASR Benchmark A comparative study and benchmark of three state-of-the-art Automatic Speech Recognition (ASR) models fine-tuned on 800 minutes of Algerian Arabic Dialect (Darija), evaluating WER and CER performance in low-resource settings. ## Overview This repository contains a comparative study and benchmark of three state-of-the-art Automatic Speech Recognition (ASR) models fine-tuned on **800 minutes** of Algerian Arabic Dialect (Darija). The goal is to evaluate the performance of different architectures in low-resource settings. ## Models Compared | Model | Architecture | Training Strategy | Parameters | | :--- | :--- | :--- | :---: | | **OpenAI Whisper (Small)** | Transformer (Seq2Seq) | Full Fine-tuning | 244M | | **Wav2Vec2-XLSR-53** | Transformer (CTC) | Full Fine-tuning | 315M | | **MMS-1B** | Transformer (CTC) | Full Fine-tuning | 1B | ## 📊 Results Summary | Model | WER | WER (+LM) | CER | CER (+LM) | Notes | | :--- | :---: | :---: | :---: | :---: | :--- | | **Whisper Small** | 49% | 49% | 17% | 17% | LM integration via **N-best Rescoring**. | | **XLSR-53** | 50% | **42%** | 16% | **15%** | Full Fine-tuning + CTC Decoding. | | **MMS-1B** | 41% | **37%** | 13% | **12%** | FULL Fine-Tuning + CTC Decoding. | > **Note:** > - For **MMS & XLSR**: LM was used during decoding (Shallow Fusion/CTC Beam Search). > - For **Whisper**: LM was used to rescore the N-best hypotheses. ## 🚀 Quick Start (Colab Notebooks) Reproduce the results or run inference directly in your browser using Google Colab. | Task | Model | Notebook | | :--- | :--- | :---: | | **Inference** | 🏆 **MMS-1B** (Best Performance) | | | **Inference** | 🥈 **Wav2Vec2-XLSR-53** | | | **Inference** | 🥉 **Whisper Small** | | > **⚠️ Important Note:** The datasets used in these notebooks are hosted privately on Hugging Face. > * The notebooks handle environment setup and data downloading automatically. > * **However, you must provide your own Hugging Face Access To …