sam4rano/yoruba_streaming_stt

Streaming and realtime transcription from speech to text # Real-Time Yoruba Speech-to-Text (ASR) System A decoupled, production-optimized real-time Automatic Speech Recognition (ASR) system for the Yoruba language. It features a Python FastAPI backend powered by the **CTranslate2 (faster-whisper)** engine, and a modern Next.js frontend built using **Tailwind CSS v4**, **shadcn UI**, and **Prisma ORM** with **PostgreSQL** database integration. --- ## 🛠️ Architecture & Workflow ```mermaid graph TD A[Browser: Mic Input] -- "16kHz PCM (WebSockets)" --> B[FastAPI Backend] B --> C[Silero VAD Segmenter] C -- "Speech Ends (asyncio.to_thread)" --> D[CTranslate2 faster-whisper Engine] D --> E[Yoruba ASR Text Output] E -- "WebSocket JSON Segment" --> A A -- "Save Action" --> F[Next.js Server Actions] F -- "Prisma ORM" --> G[PostgreSQL Database] ``` 1. **Audio Capture**: The frontend Next.js app captures microphone input via the browser's Web Audio API, downsamples it to $16\text{kHz}$ mono, converts it to 16-bit signed PCM, and streams the raw bytes over WebSockets. 2. **Asynchronous VAD & Inference**: The Python backend processes chunks through **Silero VAD**. When a speaker pause/silence is detected, inference is offloaded to a background worker thread (`asyncio.to_thread`) running **CTranslate2** (`faster-whisper`) with `int8` CPU quantization. 3. **Responsive Socket loop**: By running model inference on a separate thread, the WebSocket event loop is never blocked, allowing the server to handle concurrent user connections smoothly. 4. **Data Persistence (Prisma ORM)**: Users can save transcriptions. The frontend triggers Next.js **Server Actions** which execute secure, server-side database insertions using **Prisma ORM** connecting to any PostgreSQL database (such as Neon.tech free tier, Supabase PostgreSQL, Render, etc.). --- ## 📁 Repository Structure ```text ├── backend/ # FastAPI & CTranslate2 backend │ ├── whisper-small-yoruba-ct2/ # Compiled CTranslate2 model directory │ ├── server_production.py …