surafelml/adapt-mnmt

Dynamic Transfer Learning for Low-Resource Neural Machine Translation # Dynamic Transfer Learning for Low-Resource Neural Machine Translation __Updates__ [July, 2020] Updated repo with scripts and notes on experimental settings --- This repo implements the following papers and associated features based on OpenNMT-tf1.15: Transfer Learning in Multilingual Neural Machine Translation Adapting Multilingual Neural Machine Translation to Unseen Languages ## Experimental Settings --- #### Requirements - Mosesdecoder - SentenciePiece - See/run: `./setup-env.sh` ### Data Experiments utilize the Ted Talks data, for its low-resource nature (ranging from ~5k to ~200k parallel examples) for more than 50 languages paired with English, from Qi et al. `./scripts/get-data.sh` ### Preprocessing Prepare data for `src/s - tgt/s` pair/s (if flag is specified, tgt-lang-id is appended on the src side): `./scripts/build-training-data.sh ['src1-en en-src1 src2-en en-src2'] [flag] [exp-id]` Preprocess (clean, detokenize, and subword segmentation with sentencepiece): `./scripts/preprocess.sh [exp-id] [subword-size]` ### Pre-Training Parent Model Train a parent model, that exhibits a relatively high-resource data (e.g. Portuguese-English / `Pt-En`). `./train.sh [exp-id] [gpu-device]` ### Progressive Adaptation (ProgAdapt) to New Translation Directions --- Steps for ProgAdapt of the parent model `Pt-En` to child low-resource pair Galician-English / `Gl-En`. __Data__ `./scripts/build-training-data.sh 'gl-en' [child-model_exp-id]` __Data Preprocessing__ `./scripts/preprocess.sh [child-model_exp-id] [subword-size]` __ProgAdapt Training__ Training first customizes the parent model by taking in to consideration the child model (`Gl-En`) newly generated vocabulary: `./train-dynamic-tl.sh [parent-model_exp-id] [child-model_exp-id] [gpu-device]` ### Progressive Growth (ProgGrow) with New Translation Directions --- ProgGrow differs from progAdapt by incorporating the `Pt-En` parent model translation direction, whi …