habibadoum/lingua-franca

This project explores the use of Mistral's fine-tuning API to build a machine translation system for Sango, a Central African language with limited online resources. # Mistral AI Hackathon Project - LLM Fine-tuning for Sango Translation ## Table of Contents 1. Project Description 2. Dataset 3. Model Training 4. Results 5. Next Steps 6. Resources and Licensing 7. Contact ## Project Description This project was developed as part of the Mistral AI fine-tuning hackathon, which took place from June 5 - 30, 2024. The primary goal was to use Mistral's fine-tuning API to build a robust translation system for Sango, the lingua franca of the Central African Republic. Sango is a language with limited online resources, and this project aims to bridge the digital language gap, empowering Sango speakers across the region, fostering education, information access, and a stronger sense of community. ## Dataset The dataset used for this project was manually built and consists of 38,000 pairs of French-Sango translations. The sources for the dataset include: - The French-Sango dictionary - Personal translations - Sentences from learning websites Building this dataset was a time-consuming process due to the scarcity of online resources. The manual effort to compile and verify translations ensures a high level of accuracy and relevance. ## Model Training The model was trained on the 38,000 translation pairs over 200 steps and tested on 100 example pairs taken from the FLORES-200 benchmark. Due to limited credits, the training was constrained to 200 steps. Multiple models were trained to ensure a comprehensive approach within the given resources. The model used for fine-tuning was the `open-mistral-7b`. ## Results The performance of the model was evaluated using several metrics. Here are the results: - **BLEU:** 0.005 - **ROUGE-1:** 0.250 - **ROUGE-2:** 0.037 - **ROUGE-L:** 0.182 - **METEOR:** 0.076 - **TER:** 95.782 ### Explanation of Results The model shows promising results for some translations but struggles with complex sentences. The corpus lacked a wide range of sentences to help the model learn the nuances of the Sango language. #### …