Neuravox-Foundation/voicelink-core

Open source speech data pipeline for low resource languages. # VoiceLink An open-source speech data pipeline for low-resource African languages. VoiceLink ingests long-form audio (radio archives, live radio call-ins, community recordings), segments it into short speech clips with voice-activity detection, optionally transcribes the clips, and prepares the result for submission to open datasets such as Mozilla Common Voice. The first target languages are **Luganda** and **Ateso**, both widely spoken in Uganda and both significantly underrepresented in public speech corpora. --- ## Why this project exists Modern speech recognition and voice technologies are overwhelmingly trained on a small handful of high-resource languages — English, Mandarin, Spanish, and a few dozen more. Languages with tens of millions of speakers in East Africa (Luganda ~10M, Ateso ~2.5M) have almost no public training data, which keeps commercial speech tools unusable for those communities. VoiceLink's purpose is to reduce that gap: build a pipeline that can process **donated, consented, broadcast-quality audio** into clean, clip-level datasets, and contribute them to permissive public corpora. This is a non-commercial research and community-data project. It is run from Uganda by an independent developer. There is no end-user product, no marketing channel, no consumer-facing service. --- ## What it does, step by step 1. **Ingest** — Long-form audio enters the system from one of two sources: - **Archive ingestion** (`ingest_archives.py`) — batch upload of existing radio archives (e.g. a Luganda broadcast archive from a partner station, ~6,700 one-hour MP3s). SHA-256 deduplication, ffprobe duration, upload to Google Cloud Storage, row insert in Supabase. - **Live ingestion** (`server.py`) — a FastAPI server that receives Twilio recording webhooks from a partner radio station's live call-in line, streams the recording to GCS, and registers it in the database. 2. **Segment** (`worker/process_audio.py`) — Each long recording is downloaded, normalize …