Mark-Kitur/Swahili_English-Translator

**Swahili–English Neural Machine Translation Model** Transformer Architecture (Attention Is All You Need) This repository contains an end-to-end Swahili–English Neural Machine Translation (NMT) system implemented using the Transformer architecture introduced in the landmark paper Attention Is All You Need. The project includes dataset preprocessing, custom tokenizer creation, model definition, training pipeline, and inference utilities. The goal of this project is to build a fully functional sequence-to-sequence translation model without relying on external pretrained weights, while demonstrating a clean and reproducible implementation of the Transformer architecture. **1. Project Overview** The Transformer architecture eliminates recurrence and convolution by relying entirely on multi-head self-attention, enabling efficient parallelism and improved long-range sequence modeling. This project applies that architecture to translate Swahili sentences into English using a dataset collected from open parallel corpora. Key objectives of the project include: Build a custom tokenizer for both languages. Implement the original Transformer components from scratch. Train an encoder–decoder model following the “Attention Is All You Need” specification. Evaluate translation quality using BLEU scores. Provide an inference script for real-time translation. **3. Tokenizer Construction** A key objective of this project was to build the tokenizer manually rather than relying on prebuilt libraries. Tokenizer Design Text normalization Lowercasing Removing non-language symbols Basic punctuation handling Subword vocabulary construction Built using Byte Pair Encoding (BPE) Separate vocabularies for Swahili and English Special tokens included: , , , Vocabulary size Configurable; default is typically 8k–16k tokens per language. Encoding and decoding utilities Convert text to token IDs Convert token IDs back to text Handle unknown and padding tokens **4. Model Architecture** The mode …