yigagilbert/luganda-english-speech-builder

Builds a clean Luganda–English paired speech dataset with translation, TTS synthesis, validation, and export for multilingual speech research. # Luganda–English Paired Speech Dataset Pipeline A production-grade, modular Python pipeline for building a structured **Luganda–English** bilingual audio dataset from open-source Hugging Face resources. ## Target Schema | Column | Type | Source | |---|---|---| | `id` | `string` | Generated (`{source}_{split}_{idx:07d}`) | | `audio_lug` | `Audio(16kHz)` | Original Luganda audio from HF datasets | | `text_lug` | `string` | Original Luganda transcript | | `text_eng` | `string` | MT output — NLLB-200 or Sunbird MT | | `audio_eng` | `Audio(16kHz)` | TTS output — Sunbird TTS / SpeechT5 | --- ## Project Structure ``` luganda_pipeline/ ├── README.md ├── requirements.txt ├── setup.py ├── .env.example │ ├── config/ │ └── config.yaml # All pipeline knobs in one place │ ├── luganda_pipeline/ # Main package │ ├── __init__.py │ ├── pipeline.py # Orchestrator — runs all stages end-to-end │ │ │ ├── ingestion/ │ │ ├── __init__.py │ │ └── loader.py # Stage 1: Load & normalise HF datasets │ │ │ ├── preprocessing/ │ │ ├── __init__.py │ │ └── audio.py # Stage 2: Resample, VAD trim, normalise │ │ │ ├── filtering/ │ │ ├── __init__.py │ │ └── text.py # Stage 3: Text clean, SNR filter, dedup │ │ │ ├── translation/ │ │ ├── __init__.py │ │ └── translate.py # Stage 4: Lug→Eng via NLLB-200 / Sunbird │ │ │ ├── tts/ │ │ ├── __init__.py │ │ └── synthesize.py # Stage 5: English TTS (Sunbird / SpeechT5) │ │ │ ├── assembly/ │ │ ├── __init__.py │ │ └── build.py # Stage 6: Schema assembly & HF Hub push │ │ │ ├── qa/ │ │ ├── __init__.py │ │ └── report.py # Stage 7: QA stats, plots, dataset card │ │ │ └── utils/ │ ├── __init__.py │ ├── logging.py # Structured logger with Rich │ ├── audio_utils.py # Shared audio helpers │ └── checkpoint.py # Resume-from-checkpoint logic │ ├── …