Muhammad-Rayyan-Mohsin/mapo-reproduction

Laptop-scale reproduction of MAPO (arXiv:2401.06838) with a novel PAPC improvement for low-resource language preference optimization # MAPO Reproduction & Improvement University NLP assignment: a laptop-scale reproduction of **MAPO — Multilingual Alignment via Preference Optimization** (arXiv:2401.06838) plus a novel improvement (PAPC). > **Read `mapo-repro/EXPLAINER.md` first.** It walks through the paper, the three reproduction iterations (v1 → v2 → v3), and the proposed improvement in plain language. --- ## Repository layout ``` . ├── mapo-repro/ # Python ML pipeline (the actual reproduction + improvement) │ ├── EXPLAINER.md <- start here │ ├── configs/config.py │ ├── scripts/ <- 6-stage pipeline (translate → prefs → train → eval → report → export) │ ├── data/ <- intermediate JSONL (audit trail of what we trained on) │ ├── outputs/ <- v3 results (current); v1 and v2 frozen in subdirs │ └── logs/ └── mapo-demo/ # Next.js demo site that visualizes the framework └── lib/data/repro-results.ts <- autogenerated from outputs/results.json ``` --- ## What's in here This repo tells a three-act research story: | Version | Approach | Final train loss | Reward margin (peak) | |---|---|---:|---:| | **v1** — Answer-equality scoring | Score candidates by `2·matches_pivot + 1·matches_gold` | 0.6932 | 0.0004 | | **v2** — NLLB faithful scoring | Paper-faithful `−CE(NLLB(Y_lang → Y_en)) / len` | 0.6929 | 0.0007 | | **v3** — PAPC (our contribution) | v2 + back-translated synthetic chosen for low-resource langs | **0.6878** | **0.011** | PAPC produces a **16× larger reward margin** and a **26× larger train-loss reduction** than v2 — the metrics DPO is directly optimizing. See `mapo-repro/EXPLAINER.md` for full results, methodology, and the honest "negative-eval-with-positive-mechanism" discussion. --- ## Running the pipeline ```bash cd mapo-repro python -m venv .venv && source .venv/bin/activate pip install -r requirements.txt python scripts/01_translate.py # GSM8K → 9 non-English langs via NLLB python scripts/02_build_preferences …