femiariel/IlewaAI

Benin Multimodal Hackhaton # IlewaAI App Benin Multimodal and Multilingual Hackhaton # Project for Image Generation from Local Languages This project uses a series of technologies to transcribe speech into text, translate this text, and generate images from the translated text. The complete pipeline includes the following steps: 1. **Speech to text transcription** with MMS (Meta Speech-to-Text) 2. **Text Translation** with NLLB (No Language Left Behind). 3. **Image generation** from translated text with Stable Diffusion. 4. **Display of the generated image** with Streamlit. ## Table of Content - Installation - Utilisation - Technologies - Contribution - Licence ## Installation To run this project locally, follow the following steps: : 1. Clone this repository: : ```bash git clone github.com cd IlewaAI ``` 2. Create and activate a virtual environment: : ```bash python -m venv env source env/bin/activate # On Windows, use `env\Scripts\activate` ``` 3. Install the dependencies: : ```bash pip install -r requirements.txt ``` ## Usage 1. **Launching the application:** ```bash streamlit run ismo.py ``` 2. **User Interface:** - **Upload or record** : Upload or record an audio containing speech in the local language (Fon or Yoruba). - **Transcription** : The speech is transcribed into text using MMS. - **Translation** : The text is translated into a language supported by Stable Diffusion using NLLB. - **Image generation** : An image is generated from the translated text with Stable Diffusion. - **Display** : The generated image is displayed via the Streamlit interface. ## Technologies - **MMS (Meta Speech-to-Text)** : For the transcription of speech into text. - **NLLB (No Language Left Behind)** : For the translation of text from fon to french. - **Docker** : To containerize our Hugging Face models and deploy them efficiently. - **Azure Container Instances** : To host the APIs for each model. - **GPT-4 (Azure OpenAI)** : For the translation of text from yoruba to engl …