KevKibe/African-Whisper

🚀 Framework for seamless fine-tuning of Whisper model on a multi-lingual dataset and deployment to prod. African Whisper: ASR for African Languages *Framework for seamless fine-tuning and deploying Whisper Model developed to advance Automatic Speech Recognition (ASR): translation and transcription capabilities for African languages*. ## Features - 🔧 **Fine-Tuning**: Fine-tune the Whisper model on any audio dataset from Huggingface, e.g., Mozilla's Common Voice, Fleurs, LibriSpeech, or your own custom private/public dataset etc - 📊 **Metrics Monitoring**: View training run metrics on Wandb. - 🐳 **Production Deployment**: Seamlessly containerize and deploy the model inference endpoint for real-world applications. - 🚀 **Model Optimization**: Utilize CTranslate2 for efficient model optimization, ensuring faster inference times. - 📝 **Word-Level Transcriptions**: Produce detailed word-level transcriptions and translations, complete with timestamps. - 🎙️ **Multi-Speaker Diarization**: Perform speaker identification and separation in multi-speaker audio using diarization techniques. - 🔍 **Alignment Precision**: Improve transcription and translation accuracy by aligning outputs with Wav2vec models. - 🛡️ **Reduced Hallucination**: Leverage Voice Activity Detection (VAD) to minimize hallucination and improve transcription clarity. The framework implements the following papers: 1. Robust Speech Recognition via Large-Scale Weak Supervision : Speech processing systems trained to predict large amounts of transcripts of audio on the internet scaled to 680,000 hours of multilingual and multitask supervision. 2. WhisperX: Time-Accurate Speech Transcription of Long-Form Audio for time-accurate speech recognition with word-level timestamps. 3. Pyannote.audio: Neural building blocks for speaker diarization for advanced speaker diarization capabilities. 4. Efficient and High-Quality Neural Machine Translation with OpenNMT: Efficient neural machine translation and model acceleration. For more details, you can refer to the Whisper ASR model paper. ## Docu …