Krishna8827/speech-to-text-isixhosa

# speech-to-text-isixhosa Automatic speech recognition (ASR) pipeline for isiXhosa speech-to-text, built around OpenAI Whisper fine-tuning on the zionia/isixhosa-asr dataset. ## Requirements - Python 3.11 - FFmpeg (required by yt-dlp and librosa for audio decoding) - Optional: NVIDIA GPU with CUDA for faster training and inference On Windows, audio from Hugging Face datasets is decoded with `soundfile` (via `scripts/training_utils.py`) so training works without the optional `torchcodec` backend. ## Installation ```bash python -m venv .venv # Windows .venv\Scripts\activate # macOS / Linux source .venv/bin/activate pip install -r requirements.txt ``` ## Project Structure ``` speech-to-text-isixhosa/ ├── config.py # Shared paths, model, and training settings ├── data/ │ ├── raw/ # Downloaded source audio │ ├── processed/ # 16 kHz mono WAV files │ ├── train/ # Exported train manifests (optional) │ ├── validation/ # Exported validation manifests (optional) │ └── test/ # Exported test manifests (optional) ├── scripts/ │ ├── download_audio.py # Download YouTube audio │ ├── preprocess_audio.py # Normalize audio to 16 kHz mono WAV │ ├── dataset_builder.py # Export HF dataset splits and manifests │ ├── train.py # Fine-tune Whisper on isiXhosa ASR data │ ├── evaluate.py # Compute WER on validation/test splits │ ├── inference.py # Transcribe new WAV files │ └── training_utils.py # Shared training/evaluation utilities ├── models/ │ ├── checkpoints/ # Training checkpoints (created during training) │ └── final/ # Exported fine-tuned model ├── results/ # Evaluation and training summaries └── logs/ # Log files and TensorBoard events ``` ## Training Fine-tune `openai/whisper-small` on the Hugging Face dataset `zionia/isixhosa-asr`: ```bash python scripts/ …