cdli-gh/Semi-Supervised-NMT-for-Sumerian-English

Exploring the Limits of Low-Resource Neural Machine Translation Figure-1: Shows a Cuneiform inscription, extracted from actual tablets. Sumerian: pisan-dub-ba sza3-bi su-ga sag-nig2-gur11-ra u3 zi-ga lu2-kal-la i3-gal2 ... English: Basket-of-tablets: therefroms, restitutions, debits, and credits, of Lukalla are here; ... # Sumerian-English Neural Machine Translation As a part of the MTAAC project at CDLI, we aim to build an end-to-end NMT Pipeline while making use of the extensive monolingual Sumerian Data. Previous models that have been used to carry out English Sumerian Translation have only made use of the available parallel corpora. Presently we have only about 50K extracted sentences for both languages in the parallel corpora, whereas around 1.47M sentences in the Sumerian monolingual corpus. This huge amount of monolingual data can be used to improve the NMT system by combining it with techniques like Back Translation, Tranfer Learning and Dual Learning which have proved specially useful for Low-Resource languages like Sumerian which have a limited amount of parallel data. Moreover, we also look to implement models like XLM and MASS for the same. Requirements - Python 3.5.2 or higher - NumPy - Pandas - PyTorch - Torch Text - OpenNMT-py - fairseq Table of Contents ================= * Repository Structure * Results * Visualisations and Interpretations * Tasks: ## Repository Structure ``` |__ translation/ --> all translation models used for Sumerian-English Translation |__ transformer/ --> Supervised NMT using Vanilla Transformer |__ runTransformerSumEn.sh --> to perform training |__ README.md --> lists down all checkpoints and steps to run training and inference. |__ backtranslation/ --> fairseq usgae for Back Translation using Vanilla Transformers |__ backtranslation-onmt/ --> OpenNMT usage for Back Translation using Vanilla Transformers |__ backtranslateONMT.py --> to translate all Sumerian Text in a given shard using weights from the previous iteration |__ stack.py --> To stack the backtranslated sentenc …