joshsalako/yoruba

Machine translation from English to Yoruba # English-to-Yoruba Neural Machine Translation using MT5 This repository contains code for fine-tuning a pre-trained MT5 (Multilingual T5) model for English-to-Yoruba Neural Machine Translation. The project leverages the power of a large multilingual model to achieve high-quality translations between these languages. ## Features - **MT5 Model:** Utilizes the "google/mt5-small" pre-trained model for its multilingual translation capabilities. - **Data Preprocessing:** Includes NFC normalization of Yoruba text and filtering of single-word sentences to enhance data quality. - **Custom Dataset and Dataloader:** Implements a custom `Seq2SeqDataset` class and dataloaders for efficient data handling. - **Fine-tuning:** Fine-tunes the MT5 model on the Menyo-20k_MT dataset for English-Yoruba translation. - **Custom Trainer:** Employs a custom `CustomSeq2SeqTrainer` to ensure tensor contiguity during training and saving. - **Evaluation with BLEU:** Evaluates translation quality using the BLEU (Bilingual Evaluation Understudy) score. - **Model Saving:** Saves the fine-tuned model and tokenizer for future use. ## Dataset This project utilizes this dataset specifically designed for English-Yoruba machine translation. You can download the dataset from the provided link and place it in the `data/` directory. ## Evaluation The script will output the BLEU score on the test data after training, providing a quantitative measure of the translation quality. You can also inspect sample translations generated by the model in the output. ## Future Work - Train on a larger and more diverse dataset to improve translation accuracy and fluency. - Explore different hyperparameter settings to potentially enhance model performance. - Consider additional evaluation metrics, such as METEOR and ROUGE, for a more comprehensive assessment of translation quality. ## Contributing Contributions are welcome! If you find any issues or have suggestions for improvement, please feel free to open an …