openidle-dev/idle-scribe

Self-hosted, Afrikaans-first speech-to-text with speaker diarization — local Whisper + pyannote, an editable transcript editor, and txt/srt/vtt/json export. # idle-scribe Self-hosted, **Afrikaans-first** speech-to-text with speaker diarization. Upload an audio file in the browser and get back an accurate, timestamped transcript that knows *who said what when* — then edit it inline and export it. Transcription and diarization are deliberately independent stages that merge at the end, which keeps the transcription engine pluggable (a local Whisper model and the OpenAI API both just produce text + word timestamps; speaker labels are layered on separately). ## Features - Upload → normalize → transcribe → diarize → merge, as an async job with status polling - **Pluggable engines**, selectable per job: `faster-whisper` `large-v3` on GPU (default) or OpenAI as a fallback - **Speaker diarization** via `pyannote.audio` with word-level speaker assignment - **Editable transcript editor** (text + speaker), with audio playback - Every text correction is stored as a **training pair** (audio time range ↔ corrected text) for future fine-tuning - Per-job **language hint** (auto / Afrikaans / English) - Export to **txt / srt / vtt / json** ## Architecture ```mermaid flowchart LR subgraph Frontend["Web UI (React + Vite)"] U[Upload] --> P[Poll status] --> V[Edit + export] end subgraph Backend["FastAPI service"] API[REST API] --> JOB[(SQLite + disk)] API --> W[Async worker] W --> N[1. Normalize ffmpeg → 16k mono wav] N --> T["2. Transcribe faster-whisper / OpenAI"] N --> D[3. Diarize pyannote CPU] T --> M[4. Merge speaker per word] D --> M --> JOB end U -->|POST audio| API P -->|GET status| API V -->|GET/PUT transcript| API ``` ## Project layout ``` backend/ app/ main.py FastAPI app + routes config.py settings (env-overridable, IDLE_SCRIBE_ prefix) db.py SQLite job + corrections store worker.py single-consumer async job pipeline audio.py ffmpeg normalize cuda_setup.py Windows CUDA DLL discovery (cuBLAS/cuDNN) merge.py pure word→speaker merge export.py …