ChristianByabushi/mashi-english

# Mashi-English NMT Structured implementation for Mashi-English neural machine translation. The original notebook workflow is split into maintainable modules: - `mashi_nmt.data`: corpus loading, cleaning, splitting, and profiling. - `mashi_nmt.models`: NLLB-200, mBART-50, M2M-100, and NLLB LoRA loading. - `mashi_nmt.translation`: tokenization, generation, and bidirectional evaluation. - `mashi_nmt.training`: full fine-tuning and LoRA fine-tuning loops. - `mashi_nmt.experiments`: zero-shot and fine-tuning orchestration. - `mashi_nmt.analysis`: best/worst translation examples and sentence-level scores. - `mashi_nmt.plots`: publication-oriented figures. ## Setup ```bash pip install -r requirements.txt ``` Weights & Biases is optional. If enabled, set the key outside the code: ```bash set WANDB_API_KEY=your_key ``` ## Commands Profile the corpus: ```bash python -m mashi_nmt.cli --train-path data/corpus-mashi-english.csv profile ``` Run all zero-shot baselines: ```bash python -m mashi_nmt.cli --train-path data/corpus-mashi-english.csv zero-shot ``` Run only NLLB-200 zero-shot: ```bash python -m mashi_nmt.cli --train-path data/corpus-mashi-english.csv zero-shot --models nllb-zero-shot ``` Run only mBART-50 zero-shot: ```bash python -m mashi_nmt.cli --train-path data/corpus-mashi-english.csv zero-shot --models mbart-zero-shot ``` Run only M2M-100 zero-shot: ```bash python -m mashi_nmt.cli --train-path data/corpus-mashi-english.csv zero-shot --models m2m-zero-shot ``` Compare mBART-50 and M2M-100 zero-shot: ```bash python -m mashi_nmt.cli --train-path data/corpus-mashi-english.csv zero-shot --models mbart-zero-shot m2m-zero-shot ``` Fine-tune NLLB-200: ```bash python -m mashi_nmt.cli --train-path data/corpus-mashi-english.csv --epochs 8 --batch-size 10 fine-tune --model nllb-full ``` Fine-tune NLLB-200 with LoRA: ```bash python -m mashi_nmt.cli --train-path data/corpus-mashi-english.csv --epochs 8 --batch-size 16 fine-tune --model nllb-lora ``` Anal …