Back-Translation and Unsupervised Domain Adaptation for Machine Translation of Arabic Dialects

Despite widespread use of Dialectal Arabic, research and resources for machine translation of the various dialects remain limited. Existing systems are often restricted to handcrafted rules, while intricate similarities and differences between dialects, together with the scarcity of parallel training data, complicate the use of more modern transformer-based models. We develop and explore two methods to improve machine translation of low-resource Arabic dialects in a practical setting: back-translation to address the data scarcity problem, and unsupervised domain adaptation to utilize unlabeled data and leverage lexical similarities among different dialects. For back-translation, sentences most similar in domain to a given training set are selected by means of cosine similarity in the AraBERT embedding space. This leads to improved performance of a translation model, even when the sentences to be back-translated themselves are machine translated from a high-resource language like English. Then, with domain adaptation it is demonstrated that a model trained to translate one group of dialects (the source domain) can effectively translate other dialects (the target domain) without any labeled data of the target. The proposed method involves initially training a translation model on the source dialects, and then fine-tuning on unlabeled data of the target dialects while jointly training a domain discriminator that encourages dialect-invariant feature learning. Various combinations of the back-translation and domain adaptation methods are evaluated, in single- and multi-dialect models, and clear improvements over baselines and previous work are observed.