Zakaria831/darija-to-arabic-translator

A fine-tuned mT5 model for translating Moroccan Darija to Standard Arabic # Darija to Arabic Translation Using mT5 ## Table of Contents 1. Project Overview 2. Dataset 3. Data Preprocessing 4. Model Details 5. Training Process 6. Evaluation Metrics & Results 7. Visualizations 8. Flask Web Application - Features - Usage - Screenshot --- ## Project Overview This project focuses on **translating Moroccan Darija** into **Standard Arabic** using a fine-tuned **mT5 model**. Darija, a Moroccan dialect of Arabic, differs significantly from Standard Arabic in terms of vocabulary, grammar, and pronunciation. By leveraging the capabilities of the mT5 model, this project aims to bridge the linguistic gap, providing accurate and reliable translations from Darija to Arabic. The repository comprises two main components: 1. **Training Notebook**: Demonstrates data preprocessing, model fine-tuning, and evaluation. 2. **Flask Web Application**: Offers a user-friendly interface for real-time translation, including voice input and translation history management. --- ## Dataset - **Columns**: - `darija`: Contains text in Moroccan Darija. - `arabic`: Contains corresponding translations in Standard Arabic. **Dataset Characteristics**: - **Size**: The dataset comprises 47,185 Darija-Arabic sentence pairs. - **Quality**: Data has been meticulously cleaned to ensure accuracy and consistency. --- ## Data Preprocessing Effective preprocessing is crucial for enhancing model performance. The preprocessing steps undertaken include: 1. **Data Cleaning**: - **Removal of Missing Values**: Rows with empty `darija` or `arabic` fields are excluded to maintain data integrity. - **Deduplication**: Duplicate sentence pairs are removed to prevent the model from overfitting on repeated data. - **Normalization**: Text is standardized to ensure uniformity across the dataset. 2. **Task Prefixing**: - Each input sentence in Darija is prefixed with `"translate darija to arabic: "` to guide t …