uuklop/Twi_Chinese_Neural_Translation

Neural Machine Translation between Twi and Chinese using a modernized Transformer architecture. Features RMSNorm, SwiGLU, shared BPE tokenization, and mixed precision training. Built for low-resource African language translation with PyTorch. # Twi ↔ Chinese Neural Machine Translation A **bidirectional neural machine translation** system between **Twi** (Akan, Ghana) and **Mandarin Chinese** — two languages significantly underrepresented in NLP research. Built on a modernised Transformer trained as a single shared model for both directions simultaneously. --- ## Live Demo **Try it now:** twichi-translate.streamlit.… The web app supports bidirectional translation, adjustable beam search, a confidence score bar, and 500 example sentences per language. Built with Streamlit and served from Hugging Face Hub model weights (JamesSalar/twi-chinese). --- ## Training Results Trained for **60 epochs** on ~41 500 parallel Twi–Chinese sentence pairs (bidirectional, ~83 000 total training examples). Model selected by best average BLEU on the validation set. | Metric | Value | |---|---| | **Best Avg BLEU** (Twi↔Chi) | **26.05** (epoch 52) | | **Best Twi → Chinese BLEU** | **30.98** (epoch 47) | | **Best Chinese → Twi BLEU** | **21.53** (epoch 57) | | Final Val Perplexity | 64.5 | | Final Val Token Accuracy | 51.5% | | Final Train Token Accuracy | 93.4% | ### Training Curves Individual plots | Perplexity | Token Accuracy | |:---:|:---:| | | | | BLEU Score | Learning Rate Schedule | |:---:|:---:| | | | --- ## Architecture Based on *Attention Is All You Need* (Vaswani et al., 2017), with the following modernisations: | Component | Original | This Work | |---|---|---| | Attention | Scaled dot-product | **Flash Attention** (`F.scaled_dot_product_attention`) | | Position encoding | Sinusoidal (absolute) | **Rotary Position Embedding (RoPE)** | | FFN activation | ReLU | **SwiGLU** | | Training precision | FP32 | **Automatic Mixed Precision (AMP, FP16)** | | Model selection | Single best checkpoint | **Checkpoint averaging** (last 8 checkpoints) | | Gradient clipping | None | `clip_grad_norm_` (max = 1.0) | **Configuration:** 6 layers · 8 attention heads · d_model = 512 …