Bekafi01/english-amharic-transformer-nmt

# English-Amharic Transformer Neural Machine Translation (NMT) An end-to-end, production-ready Neural Machine Translation (NMT) system specifically engineered for **English $\leftrightarrow$ Amharic** translation, featuring a custom **Sequence-to-Sequence Transformer** implemented from scratch in PyTorch, an Ethiopic script normalization engine, Byte-Pair Encoding (BPE) tokenization, mixed-precision training, beam search decoding, ONNX Runtime acceleration, and full FastAPI / Streamlit serving. --- ## Key Highlights - **From-Scratch Transformer Architecture**: Pre-LayerNorm (Pre-LN) multi-head self-attention and cross-attention blocks built modularly with Sinusoidal Positional Encoding and 3-way weight tying. - **Ethiopic Script Normalization**: Specialized preprocessor handling Amharic homophones (ሀ/ሐ/ኀ/ሃ, ዐ/አ, ጸ/ፀ, ሠ/ሰ), Ethiopic punctuation (`፡` `።` `፤` `፥`), and numeral standardizations. - **Shared BPE Vocabulary (32k)**: A unified subword tokenizer covering Latin and Ethiopic Unicode scripts to enable joint representation and weight tying. - **Optimized Training Loop**: Mixed-precision training (`torch.cuda.amp`), Noam learning rate schedule with warmup, dynamic length-bucketed batch sampling, and label-smoothed cross-entropy loss. - **High-Performance Inference**: Configurable Beam Search decoding with length penalty and repetition penalty, plus automated export to ONNX Runtime. - **Production Serving**: Containerized FastAPI REST API (`/translate`, `/health`) and interactive Streamlit web dashboard. --- ## Project Structure ``` english-amharic-transformer-nmt/ ├── .github/ │ └── workflows/ │ └── ci.yml # GitHub Actions CI for linting & automated tests ├── configs/ │ ├── base_config.yaml # Global seed, paths, logging, hardware config │ ├── data_config.yaml # Data sources, URLs, licensing, Ethiopic normalizers │ └── model_config.yaml # Transformer dimensions, optimizer, sched …