Zikt/Lelapa-NLP-Lesotho-ASR-Task

### Lelapa NLP — Lesotho ASR Task (Product Research Scientist) A small, end-to-end evaluation of automatic speech recognition (ASR) for code-switched Sesotho-English audio. The notebook performs: - EDA (data exploration) - EMA (model benchmarking with Whisper and Wav2Vec2) - Error Analysis (slice and visualize errors) - Recommendations (data, modeling, evaluation, deployment) This repo is designed to be reproducible on Windows (CPU or CUDA). --- ## Submission-Ready Results (Latest Recorded Run) The following values come from the executed outputs saved in `product_research_scientist_task.ipynb`: - **WER (mean +- std)** - Whisper-small: `0.8886 +- 0.2628` - Wav2Vec2: `0.8652 +- 0.2090` - Difference (Whisper - Wav2Vec2): `0.0234` (not statistically significant) - **CER (mean +- std)** - Whisper-small: `0.6784 +- 0.2648` - Wav2Vec2: `0.5347 +- 0.1550` - Difference (Whisper - Wav2Vec2): `0.1437` (statistically significant) - **Paired t-tests** - WER: `t=0.982`, `p=0.3285` (not significant) - CER: `t=5.936`, `p=0.0000` (significant) Interpretation: - On this small dataset, Wav2Vec2 and Whisper are close on WER. - Wav2Vec2 is materially better on CER, indicating more stable character-level transcription. --- ## **Current Project Structure** ``` lesotho_asr_task_submit/ ├── product_research_scientist_task.ipynb # Main notebook ├── run_quick_eval.py # Script entrypoint (notebook parity) ├── README.md # Project documentation ├── requirements.txt # Dependencies ├── data/ # Audio and transcript data │ ├── audio/ # 97 .wav files │ └── transcripts/ # CSV transcripts ├── src/ # Core utilities │ ├── eval_utils.py # Evaluation utilities │ └── utils.py # General utilities ├── lm/ # Language model files …