ChristianRukundo/Kinyarwanda-Voice-Assistant
Domain:
natural language processingRecord type:
software
Creator:
Chr
Host:
# 🇷🇼 Mini Kinyarwanda Voice Assistant (using KinyaWhisper) 🤖️🎙️
Murakaza neza! Welcome to the Mini Kinyarwanda Voice Assistant project. This application demonstrates a simple voice interaction system for the Kinyarwanda language, simulating how a robot might hear, understand, and respond.
This project fulfills the assignment requirements by:
- 👂 **Hearing:** Using the `benax-rw/KinyaWhisper` model for Kinyarwanda Speech-to-Text (ASR).
- 🧠 **Understanding:** Matching the transcribed text to predefined questions using basic dictionary lookup (NLP).
- 🗣️ **Speaking:** Generating spoken Kinyarwanda answers using Google Text-to-Speech (TTS).
## 🎯 Project Goal
To build a functional prototype showcasing core voice AI components (ASR, NLP, TTS) for Kinyarwanda, suitable for demonstrating basic voice interaction in applications like robotics.
## ✨ How It Works: Code Breakdown
The `src/app.py` script orchestrates the entire process:
1. **Initialization & Model Loading:**
- Imports necessary libraries (`gradio`, `gtts`, `torch`, `torchaudio`, `transformers`).
- Defines constants like the Hugging Face `MODEL_ID` (`benax-rw/KinyaWhisper`).
- Loads the `WhisperProcessor` and `WhisperForConditionalGeneration` model from Hugging Face, automatically downloading them if needed. It detects if a GPU (`cuda`) is available for faster processing.
```python
# Loads processor and model
processor = WhisperProcessor.from_pretrained(MODEL_ID)
model = WhisperForConditionalGeneration.from_pretrained(MODEL_ID).to(device)
model.eval() # Sets model to evaluation mode
```
2. **👂 Speech Recognition (ASR - `transcribe_kinyarwanda` function):**
- Takes the audio file path as input.
- Loads the audio using `torchaudio.load()`.
- Converts stereo audio to mono.
- Resamples the audio to the required `TARGET_ASR_SAMPLE_RATE` (16000 Hz) if necessary.
- Uses the `processor` to prepare the audio features for the model.
- Feeds the features into the `model.generate()` method to get predicted token …