mukhal/low-resource-seq-labeling

[EACL 2021] Self-training Pretrained LMs for Zero- and Few-shot Arabic Sequence Labeling ## Low-resource Multi-dialectal Arabic Sequence Labeling (NER and POS tagging) #### Code for EACL 2021 paper Self-Training Pre-Trained Language Models for Zero- and Few-Shot Multi-Dialectal Arabic Sequence Labeling ## Requirements Please make sure you have `pytorch >=1.4` and `fairseq >= 0.9` installed. ## Datasets The `data/` folder includes some of the datasets used in the paper (Some of the datasets can only be accessed through the LDC). ### NER * `NER/twitter`: social media NER dataset from (Darwish, 2013) * `NER/twitter.norm`: same dataset but normalized. * `NER/ANERCorp`: MSA dataset obtained from (Benajiba et al., 2007) * `NER/zero-shot-dialect`: this is the zero-shot dialectal setting. training data is from ANERCorp (Benajiba et al., 2007) while validation and test sets come from the dialectal portion of the Twitter data (Darwish, 2013). * `NER/zero-shot-msa`: same as above but validation and test sets come from the MSA portion of the Twitter data. ### POS tagging * `POS-tagging/egy`, `POS-tagging/glf`, `POS-tagging/lev`, and `POS-tagging/msa` dialectal POS tagging datasets obtained from (Darwish et al., 2018) * `POS-tagging/zero-shot-*`: training data is MSA, development, and test data from dialects. ### Unalebeled data These are unlabeled examples used for self-training. * `unlabeled_aoc`: Unlabeled AOC tweets taken from (El Araby and Mageed, 2018). ## Setting Up the Data ### NER Format your data using IOB format with a token per line and an empty line separating sentences. For example: ``` لنزار B-PERS عدد O كبير O من O الكتب O النثرية O أهمها O ``` ### POS Tagging Similar to NER: ``` و CONJ هو PRON في PREP محام NOUN TB TB ين NSUFF ``` Typically each data folder has 3 files: `train.txt`, `valid.txt`, and `test.txt` ## XLMR models We first need XLM-R models. You can donwload XLM-R models using the following commands ``` mkdir pretrained_models wget dl.fbaipublicfiles.com # bas …