imranboutadghart/speech-to-speech

working on developping a speech to speech model in Darija arabic # Speech-to-Speech: Darija Conversational AI This project is a full speech-to-speech conversational pipeline for Moroccan Darija. It transcribes speech, generates a natural language response, and synthesizes the reply as speech—all in Darija. The system uses state-of-the-art models for speech recognition, language generation, and text-to-speech. ## Overview - **Speech-to-Text (STT):** Fine-tuned Whisper model for Darija transcription. - **Language Model (LLM):** Arabic conversational LLM (AceGPT-7B or fallback to AraGPT2). - **Text-to-Speech (TTS):** DarijaTTS model for high-quality speech synthesis. - **Web Interface:** Simple frontend for uploading audio and listening to responses. ## Setup Instructions 1. **Clone the repository:** ```bash git clone github.com cd speech-to-speech ``` 2. **Install dependencies:** Ensure you have Python 3.8+ and `pip` installed. ```bash pip install -r requirements.txt ``` 3. **Run the server:** ```bash python app.py ``` 4. **Open the web interface:** Open `index.html` in your browser or access the server at localhost if running locally. ## Usage 1. Record or upload a Darija audio file (supported: wav, mp3, ogg, m4a, flac). 2. The backend transcribes, generates a response, and returns synthesized speech. 3. Listen to the AI-generated reply in Darija. ## Project Structure - `app.py` — Flask backend for the speech-to-speech pipeline - `index.html` — Web frontend for user interaction - `requirements.txt` — Python dependencies - `finetunes/` — Notebooks for model fine-tuning - `whisper.ipynb` — Fine-tuning Whisper for Darija STT - `outetts.ipynb` — Fine-tuning Outetts for Darija TTS - `outputs/` — Example output audio files ## Notes - For best results, use a machine with a CUDA-capable GPU. - The TTS model uses `outetts==0.3.3`.