EvansNjogu/NeuralMachineTranslation

Swahili-English Neural Machine Translation using Seq2Seq with LSTM. Trained on parallel corpora, evaluated with BLEU scores and tested for accuracy. Results show moderate BLEU but low exact-match accuracy, highlighting areas for improvement. # Swahili-English Machine Translation Using Seq2Seq This task implements a Swahili-English neural machine translation (NMT) model using a Seq2Seq (Encoder-Decoder) architecture with LSTM. The dataset is sourced from various Swahili-English corpora, including Tatoeba. The model achieves moderate BLEU scores but exhibits low accuracy, indicating areas for improvement. ## Project Overview - Implemented a Neural Machine Translation (NMT) model. - Utilized a Seq2Seq framework with LSTM-based encoder and decoder. - Trained on Swahili-English parallel sentence pairs. - Evaluated using BLEU scores and accuracy metrics. ## Architecture The model follows a standard Encoder-Decoder framework: 1. **Encoder**: A bidirectional LSTM processes the source sentence. 2. **Decoder**: A unidirectional LSTM generates the translation. 3. **Seq2Seq Coordination**: - Uses **Teacher Forcing** during training. - Decodes one word at a time during inference. ## Dataset The dataset includes Swahili-English sentence pairs extracted from multiple sources: - **Source Language**: Swahili (`swh`) - **Target Language**: English (`eng`) - **Total Sentences**: ~4,293 - **Filtered Dataset**: Kept sequences ≤10 words long. ### Preprocessing Steps: - Tokenization using `nltk.tokenize.WordPunctTokenizer()`. - Replaced rare words with ` ` token. - Saved processed vocabulary for training. ## Training Process - **Optimizer**: Adam (`lr=0.001`) - **Loss Function**: CrossEntropyLoss - **Batch Size**: 100 - **Epochs**: Early stopping used (~65 epochs max) - **Training Speed**: GPU-enabled for acceleration. ### Loss History The loss consistently decreased, indicating model learning. ### Evaluation Metrics The model was evaluated using: - **BLEU (Bilingual Evaluation Understudy)**: Measures n-gram overlap between predictions and references. - **Accuracy**: Percentage of exact matches (low in this case). ### Final Scores ``` BLEU-1: 0.2371 BLEU-2: 0.0819 BLEU-3: 0.0421 BLEU-4: 0.0242 Accuracy: 0.0062 ``` - …