JairusTheAnalyst/English_Swahili_Translation

# English-Swahili Translation Model A comprehensive machine translation project that fine-tunes a Helsinki-NLP transformer model for English to Swahili translation, complete with evaluation, deployment, and a web interface. ## 📋 Project Overview This project demonstrates the complete pipeline for building a neural machine translation system: - **Data Preprocessing**: Cleaning and preparing parallel English-Swahili text data - **Model Fine-tuning**: Using Helsinki-NLP's opus-mt-en-sw model with TensorFlow - **Evaluation**: BLEU score assessment using SacreBLEU metric - **Deployment**: Interactive web interface using Gradio ## 🚀 Features - **Pre-trained Model**: Helsinki-NLP/opus-mt-en-sw (University of Helsinki) - **Fine-tuning**: Custom training on parallel corpus - **Evaluation**: SacreBLEU scoring for translation quality - **Web Interface**: Gradio-based translation app - **Performance Comparison**: Before/after fine-tuning metrics ## 📊 Dataset - **Size**: 8,492 parallel sentence pairs - **Source**: Combined English-Swahili translation corpus - **Preprocessing**: - Lowercasing - Apostrophe removal - Digit removal - Train/validation split (80/20) ## 🛠️ Installation & Requirements ```bash # Core dependencies pip install transformers[sentencepiece] pip install datasets pip install evaluate pip install sacrebleu pip install sacremoses # TensorFlow for training pip install tensorflow # Web interface pip install gradio # Additional utilities pip install pandas numpy tqdm ``` ### Hardware Requirements - **GPU**: Recommended for training (CUDA-compatible) - **RAM**: 8GB+ for model loading - **Storage**: 2GB+ for model checkpoints ## 🎯 Model Architecture - **Base Model**: Helsinki-NLP/opus-mt-en-sw - **Framework**: TensorFlow/Keras - **Task**: Sequence-to-sequence translation - **Max Length**: 128 tokens - **Batch Size**: 32 (training), 16 (validation) ## 📈 Training Configuration ```python # Training parameters num_epochs = 2 learning_rate = 5e-5 weight_de …