hunterschep/nllb-200-600m-finetuning

A Colab notebook for fine-tuning NLLB-200 on low resource languages # Fine-Tuning NLLB-200 for a New Language in 2025 **Tags:** `nllb-200` `machine-translation` `fine-tuning` `low-resource-languages` `transformers` `seq2seq` `atayal` `formosan` `multilingual-nlp` --- ## Overview A practical, **2025-updated tutorial** for fine-tuning Meta's NLLB-200 machine translation model on new, low-resource languages. This is an updated successor to David Dale's classic tutorial, with **bi-directional training support**, modern Transformers APIs, and single-GPU Colab compatibility. This notebook demonstrates the full workflow using **Atayal ↔ English** translation from the FormosanBank project, but the approach generalizes to any low-resource language pair. **Key improvements over the original:** - ✅ Correct bi-directional language ID wiring - ✅ Works with Transformers ≥ 4.38 (original code is now broken) - ✅ Modern `generate()` API with `max_new_tokens` - ✅ Single-GPU T4 Colab-friendly implementation - ✅ Proper tokenizer merging (SentencePiece + NLLB vocab) --- ## Quick Links 📖 **Read the full blog post on Medium** --- ## What's Covered 1. **Data Loading** — Load parallel corpora from FormosanBank (or your own dataset) 2. **Tokenizer Extension** — Train SentencePiece, merge into NLLB, add new language codes 3. **Bi-Directional Training** — Fine-tune both directions in a single training loop 4. **Generation & Evaluation** — Generate with correct language IDs, compute BLEU/chrF 5. **Saving** — Export model + tokenizer for reuse or Hub sharing --- ## Getting Started ### Google Colab (Recommended) Just click the Colab badge above — no setup required! ### Local Setup ```bash git clone github.com cd nllb-200-600m-finetuning pip install "transformers>=4.38,<5.0" sentencepiece sacremoses sacrebleu datasets torch jupyter notebook NLLB_200_MT.ipynb ``` --- ## Citation If you use this tutorial or code, please cite: ```bibtex @misc{scheppat2025nllb, title={Fine-Tuning NLLB-200 for a New …