Samiratu/limbum_translation

Low-resource Limbum–English machine translation using Transformer baselines and NLLB, # Limbum–English Machine Translation Embedding initialization for unseen languages in multilingual NMT, applied to Limbum (`lmp`) — a Grassfields Bantu language of Cameroon with ~400,000 speakers and no prior computational resources. ## Overview We fine-tune NLLB-200-distilled-600M for Limbum→English translation using a newly assembled parallel corpus of **8,837 sentence pairs** from New Testament text and a bilingual dictionary. Our key contribution is a simple **embedding averaging** method: we add a new language token `lmp_Latn` whose embedding is initialized as the mean of three related Bantu language embeddings (Swahili, Luganda, Lingala) already in the model. This eliminates the need to choose a single proxy language. ### Results | System | Description | chrF2++ | BLEU | |--------|-------------|---------|------| | S0 | NLLB-200 zero-shot | 12.52 | 0.66 | | S1 | Transformer from scratch | 14.47 | 1.32 | | S2a | NLLB + Swahili proxy | 47.27 | 30.56 | | S2b | NLLB + averaged embedding (ours) | 46.70 | 29.88 | The averaged initialization matches the single-proxy approach (overlapping 95% CIs), while multilingual transfer accounts for a **32-point chrF2++** improvement over training from scratch. ## Project Structure ``` src/ preprocessing.py # Data loading, cleaning, stratified splits data_loader.py # PyTorch dataset classes evaluation.py # chrF2++, BLEU, diacritic accuracy, bootstrap CIs train_transformer.py # S1: Transformer from scratch train_nllb.py # S2a/S2b: NLLB fine-tuning with proxy/averaged tokens augmentation.py # Data augmentation utilities inference.py # Generate translations from saved models notebooks/ colab_experiments.ipynb # Full experiment runner (designed for Google Colab) data/ processed/ # Cleaned parallel corpus (CSV) results/ *.json # Evaluation results for all experiments ``` ## Quick Start ### Run on Google Colab (recommended) 1. Upload this repo as …