antonisa/inflection

Morphological Inflection for Low-Resource Languages using cross-lingual transfer # A Tool for Morphological Inflection This is the code accompanying our paper on "Pushing the Limits of Low-Resource Morphological Inflection". # Requirements Are listed in `requirements.txt` so you can just run ~~~ pip install -r requirements.txt ~~~ Also, run `make` in order to build the executable needed for data hallucination. # Data Hallucination Use the `augment.py` script as follows: ~~~ python augment.py [data_directory] [language] --examples N [--use_dev] ~~~ The script assumes data in the format of the SIGMORPHON 2019 task 1 inflection shared task (example from Asturian): ~~~ meyorar meyoraría V;1;SG;COND firir firir V;NFIN algamar algamareis V;2;PL;SBJV;PST;IPFV;LGSPEC1 ... ~~~ All scripts assume that files named `language-train`, `language-dev`, and `language-test` are under `data_directory`. The output is a file `language-hall` under `data_directory` with `N` hallucinated examples. If you want to also use the dev dataset for hallucination (recommended for extremely low-resource cases) add the `--use_dev` flag. ### Attribution The `align.py`, `align.c` and `Makefile` are taken from Roee Aharoni's work: github.com # Training inflection models The main script is `inflection.py` which implements the models and handles training, testing, etc. For standard training using cross-lingual transfer, run: ~~~ py inflection.py \ --datapath sample-data/ \ --L1 adyghe \ --L2 kabardian \ --mode train \ --setting original ~~~ Running the above command trains for about 40 minutes on a single CPU (2.4 GHz), producing the following output ~~~ [dynet] random seed: 2846648232 [dynet] allocating memory: 512MB [dynet] memory allocation done. Data lengths transfer-language 10000 10000 10000 ... ... Accuracy good enough, breaking [lr=0.1 clips=3401 updates=14115] Epoch 0 : 28131.39509539181 COPY Accuracy: 0.95 average edit distance: 0.05 TASK Accuracy: 0.68 average edit distance: 0.6 ... ... Ep …