keleog/PidginUNMT

Unsupervised Neural Machine Translation from West African Pidgin (Creole) to English without a single parallel sentence # Unsupervised Neural Machine Translation from West African Pidgin (Creole) to English This repository contains the implementation of an Unsupervised NMT model from West African Pidgin (Creole) to English without using a single parallel sentence during training. Link to paper - PidginUNMT: Unsupervised Ne… (Accepted at NeurIPS 2019 Workshop on Machine Learning for the Developing World) Key results: 1. The alignment of Pidgin word vectors with English word vectors which achieves a Nearest Neighbor accuracy of **0.1282**. Significantly better than a baseline of **0.0093** which is the probability of selecting the right nearest neighbor from the evaluation set of 108 pairs. This aligned vector will be helpful in the performance of various downstream tasks and transfer of models from English to Pidgin. 2. The creation of an Unsupervised Neural Machine Translation model between Pidgin and English which achieves a BLEU score of **7.93** from Pidgin to English and **5.18** from English to Pidgin on a test set of 2101 sentence pairs. Test set was obtained from the JW300 dataset [4] and preprocessed by the Masakhane group here For our results, at each training step, we performed the following: - Discriminator training to constrain encoder to map both languages to the same latent space - Denoising autoencoder training on each language which acts as language model training - On-the-fly back translation and reconstruction on sentences which acts as translation training Below is the algorithm for our training process: We trained for 8 epochs on a V100 (approx. 3 days) and selected the best model from evaluating on our test dataset. Below are some translations by our model: **The Good** | Source (language ID)| Model Translation | Target (language ID) | | --------------------------------------|:-----------------:| ---------------------:| | 1. afta dem cancel dia first attempt (pd)| after they cancelled their first attempt | after they cancelled their first att …