tushar117/XAlign

Cross-lingual Fact-to-Text Alignment and Generation for Low-Resource Languages # XAlign: Cross-lingual-Fact-to-Text-Alignment-and-Generation-for-Low-Resource-Languages In this work, we propose the creation of cross-lingual fact-to-text dataset, `XAlign` accepted at WebConf-2022 poster-demo track. It consist of English WikiData triples/facts mapped to sentences from low resources Wikipedia. We explored two different unsupervised methods to solve cross-lingual alignment task based on: - Transfer learning from NLI - Distant supervision from another English-only dataset This repository consists of steps for executing the cross-lingual alignment approaches and finetuning mT5 for data-to-text generation on XAlign. One can find more details, analyses, and baseline results in our paper. ## Installation Install the required packgaes as follow: ``` pip install -r requirements.txt ``` ## Dataset ### Dataset releases - v2.0 (Sep 2022): Extended to four additional languages: Punjabi (pa), Malayalam (ml), Assamese (as) and Oriya (or). - v1.0 (Apr 2022): Introduced the cross-lingual data-to-text into 8 langauges: Hindi (hi), Marathi (mr), Gujarati (gu), Telugu (te), Tamil (ta), Kannada (kn), Bengali (bn), and monolingual dataset in English (en). ### Data Fields Each record consist of the following entries: - sentence (string) : Native language wikipedia sentence. (non-native language strings were removed.) - `facts` (List[Dict]) : List of facts associated with the sentence where each fact is stored as dictionary. - language (string) : Language identifier. The `facts` key contains list of facts where each facts is stored as dictionary. A single record within fact list contains following entries: - subject (string) : central entity. - object (string) : entity or a piece of information about the subject. - predicate (string) : relationship that connects the subject and the object. - qualifiers (List[Dict]) : It provide additional information about the fact, is stored as list of qualifier where each record is a dictionary. The dictionary contains two …