manasseh-zw/h400-shona-speech

# HIT400 Capstone Project: Shona Speech-to-Speech System Welcome to the unified source repository for the Shona Speech-to-Speech (S2S) translation and orchestration system. This repository consolidated the entire engineering pipeline, from raw dataset curation and cleaning to fine-tuning state-of-the-art acoustic and text-to-speech models, concluding with a real-time web application showcase. --- ## 🛠️ System Architecture The capstone project is composed of four logical pipelines, organized modularly: ```mermaid graph TD A[Raw Shona Audio & Text] --> B[data-pipeline] B -->|Normalized Datasets| C[wav2vec2-bert-asr] B -->|Cleaned Transcripts & Audio| D[f5-tts] C -->|Shona Speech Recognition API| E[s2s-application] D -->|Shona Speech Synthesis API| E E -->|Real-time Orchestration| F[Client-Server Web App Interface] ``` 1. **`data-pipeline/`**: The data curation, speaker classification, volume normalization, and audio-text alignment module. 2. **`wav2vec2-bert-asr/`**: Acoustic training scripts for fine-tuning the Wav2Vec2-BERT and Whisper models on Shona voice datasets. 3. **`f5-tts/`**: Text-to-Speech fine-tuning configurations, Shona tokenizer design, benchmark, and inference implementations for F5-TTS. 4. **`s2s-application/`**: Real-time end-to-end orchestration client (React/Vite) and backend server (Bun) facilitating ultra-low latency voice translation using LiveKit. --- ## 📂 Repository Structure Below is the directory structure of this consolidated source submission: ```text h400-shona-speech/ ├── README.md # Master system documentation (this file) ├── data-pipeline/ # Data cleaning, speaker verification & curation │ ├── pyproject.toml # Dependency definitions (uv format) │ ├── uv.lock # Lockfile for dependency integrity │ ├── README.md # Data-pipeline documentation │ └── src/ # Audio engineering and ingestion source files ├── wav2vec2-bert-asr/ # ASR acoustic t …