trinitron88/ChinTranslator

Audio transcriber for Hahka-Chin language low resource language # Hakha Chin Speech-to-Text Translator A fine-tuned Whisper model for transcribing Hakha Chin (`cnh`) speech and translating it to English. Built to help bridge language barriers in Hakha Chin-speaking communities. ## 🎯 Overview **Current status: V6** — a LoRA fine-tune of `openai/whisper-large-v3-turbo` trained on the Common Voice Hakha Chin dataset (community-recorded, pre-aligned utterances with validated transcripts). Earlier versions (V1–V4) trained on Bible audio; that data and its alignment pipeline are retired to `archive/` — Common Voice gives cleaner alignment, more speakers, and conversational vocabulary. V6 is an accuracy push for **conversational** speech: a fixed surrogate language token (matching train and inference prompts), all of train+dev plus mined unvalidated clips as training data, waveform + SpecAugment augmentation, broader LoRA, best-checkpoint selection, and WER measured on the held-out official test split (158 unseen speakers). Rationale and results plan: V6_PLAN.md. There are three ways to use the model: | App | What it does | |-----|--------------| | `gradio_interface.py` | Batch: upload/record audio → Chin transcript + English translation + spoken English | | `realtime.py` | Streaming prototype: phone mic → GPU backend → English in your earbud a few seconds behind the speaker (see REALTIME.md) | | `hf_space/` | The realtime app packaged for Hugging Face Spaces (WebRTC + TURN work there; Colab can't carry WebRTC media) | Translation is Google Translate's endpoint called directly with the source pinned to `cnh` (deep-translator's language list lacks Hakha Chin, and autodetect misreads it). TTS is gTTS. ## 🚀 Quick start ```bash git clone github.com cd ChinTranslator # Serve the batch app (downloads stock large-v3 if CHIN_MODEL is unset) python gradio_interface.py # Serve the fine-tuned model (after training + export, see below) CHIN_MODEL=whisper-cnh-turbo-ct2 python gradio_interface.py ``` …