A voice-enabled Kinyarwanda assistant that uses a fine-tuned Whisper model for automatic speech recognition (ASR), rule-based NLP with fuzzy matching, and offline text-to-speech (TTS). Built for the Intelligent Robotics course at Rwanda Coding Academy, it demonstrates real-time and batch voice interaction in Kinyarwanda.
🗣️ KinyaWhisper – Kinyarwanda Voice Assistant
A lightweight, voice-enabled assistant that understands and responds in Kinyarwanda, simulating humanoid robot voice interaction. Built for the Intelligent Robotics course at Rwanda Coding Academy, this project leverages Automatic Speech Recognition (ASR), Natural Language Processing (NLP), and Text-to-Speech (TTS) to create an AI-powered assistant tailored for Kinyarwanda speakers.
🎯 Project Overview
KinyaWhisper is designed to:
🎤 Transcribe Kinyarwanda speech using a fine-tuned OpenAI Whisper model.
🧠 Understand questions via rule-based NLP with fuzzy matching.
🗣️ Respond in Kinyarwanda using offline TTS (pyttsx3).
This project demonstrates the potential of localized AI for voice interaction in underrepresented languages like Kinyarwanda.
📁 Project Structure
.
├── audio/ # Custom Kinyarwanda audio samples (44 WAV files)
├── dataset.jsonl # Metadata for training (audio paths + transcriptions)
├── kinya-whisper-model/ # Fine-tuned Whisper model
├── train.py # Script for fine-tuning Whisper
├── inference.py # Script for batch transcription
├── main.py # Batch-mode voice assistant (ASR + NLP + TTS)
├── bach_main.py # CLI for live audio recording and response
├── transcriptions.txt # Output transcriptions from inference
├── README.md # Project documentation
🚀 Features
ASR: Converts Kinyarwanda speech to text using a fine-tuned Whisper model.
NLP: Matches transcribed text to predefined answers using fuzzy logic (difflib).
TTS: Generates spoken responses in Kinyarwanda via pyttsx3.
Live Recording: Supports real-time voice input via microphone (bach_main.py).
Batch Processing: Processes multiple audio files for transcription and response (main.py).
🛠️ Technologies
Python 3.10+
OpenAI Whisper (fine-tuned whisper-small)
Transformers (Hugging Face)
Torchaudio for audio processing
Pyttsx3 for offline TTS
Sounddevice for live microphone input
Difflib for fuzzy text matching
🧠 Model Training
Dataset: 44 custom Kinyarwanda …