devalade/whisper-yoruba

Local-first Yoruba voice-query pipeline (Whisper ASR + diacritic restoration + NLLB + RAG + MMS-TTS) on Apple Silicon, with LoRA fine-tuning for whisper-large-v3 # Yoruba Voice Query Pipeline End-to-end voice assistant for Yoruba speakers. Speak a question in Yoruba, get a spoken Yoruba answer grounded in an English Wikipedia corpus. Runs fully locally on Apple Silicon — no cloud calls after the initial model downloads. ``` WAV (yo, 16 kHz mono) │ ▼ ┌──────────────┐ raw Yoruba text │ M1 ASR │ ─ Whisper Large v3 (mlx-whisper) └──────────────┘ │ ▼ ┌──────────────┐ diacritized Yoruba │ M2 ADR │ ─ Davlan/mT5_base_yoruba_adr └──────────────┘ │ ▼ ┌──────────────┐ English query │ M3 YO→EN │ ─ NLLB-200 distilled-600M └──────────────┘ │ ▼ ┌──────────────┐ English answer │ M4 RAG │ ─ MiniLM-L6 + FAISS + Mistral-7B Q4 (llama.cpp) └──────────────┘ │ ▼ ┌──────────────┐ Yoruba answer audio │ M5 TTS │ ─ NLLB EN→YO → M2 diacritize → MMS-TTS-yor └──────────────┘ │ ▼ WAV (yo) ``` M2 runs twice on purpose: once after ASR to clean the input for translation, and once inside M5 to clean the NLLB EN→YO output before TTS. ## Requirements - Apple Silicon Mac (M1/M2/M3/M4). Tested on M4 Pro / 24 GB. - macOS with Miniforge (ARM64), Python 3.11. - ~15 GB free disk for model weights. - First run downloads several gigabytes from Hugging Face. ## Setup ```bash # 1. Create env conda create -n yoruba python=3.11 -y conda activate yoruba # 2. Install deps make install # or: pip install -r requirements.txt # 3. Drop the Mistral GGUF into models/ # Expected file: models/mistral-7b-instruct-v0.2.Q4_K_M.gguf # Source: TheBloke/Mistral-7B-Instruct-v0.2-GGUF on Hugging Face mkdir -p models # (download mistral-7b-instruct-v0.2.Q4_K_M.gguf into models/) # 4. Build the Wikipedia FAISS index used by M4 make index # or: python -m scripts.build_index # writes data/wikipedia/faiss.index and data/wikipedia/passages.jsonl ``` The seed corpus (in `scripts/build_index.py`) covers Yoruba people, language, religion, Lagos, Nigeria, Ile-Ife, Oyo Empire, Wole Soyinka, Fela Kuti, Olumo Rock …