hashbuilder60-netizen/african-llm-video-translation

# African Multilingual Video-Based LLM A multimodal AI pipeline that transcribes video speech and translates it into 15 African and international languages — including the first documented AI translation and speech synthesis system for **Ibibio**, a Benue-Congo language spoken by ~10 million people in Akwa Ibom State, Nigeria. Final Year Project — Group 14, Department of Computer Science, Kwara State University, Malete. --- ## What it does ``` Video → Frame/Audio Extraction → Whisper ASR → Text Processing → Multimodal Fusion → Summarisation → NLLB-200 Translation (15 languages) → Ibibio Fine-tuned Translation → Text-to-Speech → Timestamped Subtitles ``` - **15 output languages**: Yoruba, Hausa, Igbo, Ibibio, Swahili, Amharic, Zulu, Xhosa, Twi, Wolof, Somali, Kinyarwanda, Luganda, Chichewa, French, Arabic - **Ibibio fine-tuning**: NLLB-200 distilled 600M fine-tuned on an original 438-pair Ibibio-English corpus (chrF 4.15, METEOR 12.14, from a zero baseline) - **Real voice synthesis**: integrates recorded Ibibio voice clips from the IbibioVoice dataset, with gTTS and phoneme-synthesis fallbacks - **Segment-level subtitles**: every Whisper transcript segment is translated individually with its original timestamp, exported as standard and bilingual SRT files - **Research finding**: documents catastrophic forgetting in Hausa (chrF −34.60) following Ibibio fine-tuning Full results and methodology are in the accompanying research paper. --- ## Installation **Requirements:** Python 3.10+, FFmpeg installed and on your system PATH. ```bash git clone github.com cd african-llm-video-translation python -m venv venv venv\Scripts\activate # Windows # source venv/bin/activate # macOS/Linux pip install -r requirements.txt ``` > First run will download several HuggingFace models (NLLB-200, XLM-RoBERTa, Whisper) — this requires an internet connection and several GB of disk space, cached locally …