MrHeart/idoma-english-bidirectional-tts-stt

Developing a Bidirectional English-Idoma Speech-to-Text (STT) and Text-to-Speech (TTS) System for Enhanced Communication and Language Preservation 📖 **Abstract** Most speech technology exists for high-resource languages, leaving low-resource languages like Idoma digitally underrepresented. This project bridges that gap by developing a fully functional Bidirectional English-Idoma Speech System. The system utilizes an Applied Research Design to solve data scarcity and tonal complexity issues inherent in the Idoma language. It integrates three fine-tuned transformer models to enable natural conversation flow: - **ASR (Automatic Speech Recognition):** Fine-tuned Wav2Vec 2.0 XLS-R. - **NMT (Neural Machine Translation):** Fine-tuned NLLB-200. - **TTS (Text-to-Speech):** Fine-tuned VITS-based MMS-TTS. **Note: This repository contains just the modularized version of the inference code found in the Jupyter Notebooks. For full implementation codes, check the Juypter Notebooks.** **Jupyter Notebooks** - wav2vec2-xls-r-1b-finetuned-idoma - idoma-mms-tts-eng - idu-eng-translator - Inference code for the bidirectional TTS, STT and NMT 🏗️ **System Architecture** The application follows a Service-Oriented Architecture (SOA) where the UI is decoupled from the inference logic. The processing pipeline for English → Idoma (and vice-versa) follows this flow: ```mermaid graph LR A[Input Audio] --> B[ASR Service] B --> C[NMT Service] C --> D[TTS Service] D --> E[Synthesized Speech] ``` **Core Components** - **Speech-to-Text (STT):** Converts input audio to text using Wav2Vec2 (Idoma) and Whisper Large v3 (English). - **Machine Translation (NMT):** Translates text between English and Idoma using NLLB-200. - **Text-to-Speech (TTS):** Synthesizes the translated text into speech using VITS (Idoma) and SpeechT5 (English). 📊 **Performance Results** The system was rigorously tested against a held-out test set. Below are the quantitative evaluation metrics reported in the thesis: | Component | Model Architecture | Metric | Score | | --------------|---------------------------|-------------- …