Amrith182007/kaggle-deep-past-challenge-akkadian-mt

Low-resource Akkadian → English neural machine translation using NLLB, LoRA fine-tuning, and data-centric NLP engineering for the Kaggle Deep Past Challenge. # 🏺 Deep Past Akkadian MT > Low-resource **Akkadian → English Neural Machine Translation** using **NLLB-200**, **LoRA fine-tuning**, and **data-centric preprocessing** for the Kaggle Deep Past Challenge. --- ## 📖 Overview This project explores neural machine translation for one of the most challenging low-resource language tasks: translating **4,000-year-old Old Assyrian Akkadian cuneiform transliterations** into English. Built for the **Kaggle Deep Past Initiative Challenge**, the system fine-tunes Meta's **NLLB-200** models using: - sentence-level corpus reconstruction - Akkadian text normalization - constrained decoding - TPU-based LoRA fine-tuning - custom preprocessing pipelines The project focuses heavily on **data-centric NLP engineering**, tokenizer diagnostics, and low-resource MT robustness. --- ## ✨ Key Features - 🏛️ **Ancient Language MT** Translation of Akkadian cuneiform transliterations (1950–1750 BCE) into English. - 📊 **Data-Centric Pipeline** Expanded training data from ~1,561 document-level pairs to **5,700+ sentence-level pairs** using corpus reconstruction and OARE alignment. - 🔧 **LoRA Fine-Tuning** Parameter-efficient adaptation (`r=32`) on both **NLLB-200 600M** and **1.3B** models. - 🐛 **Critical Tokenizer Bug Fix** Diagnosed and fixed a silent ` ` token decoding issue worth **+1.74 leaderboard points**. - 🔍 **Offline-Compatible Group Beam Search** Pinned `transformers==4.38.2` to preserve native Group Beam Search support for Kaggle offline inference. - ⚡ **Custom TPU v5e-8 Training Loop** Replaced HuggingFace Trainer with a custom PyTorch + XLA loop to avoid TPU deadlocks. - 📝 **Constrained Decoding** Implemented lexicon-based proper noun forcing with N-best reranking. --- ## 📈 Results | Model | Public LB | Private LB | Val BLEU | Val chrF++ | Val Geo Mean | |---|---|---|---|---|---| | NLLB-200 600M (baseline) | 20.84 | 20.24 | — | — | — | | NLLB-200 600M + pipeline fixes | 24.95 | 24.54 | 20.00 | 43.66 | 29.55 | | **NLLB- …