Andrews2017/KINNEWS-and-KIRNEWS-Corpus

Data, Embeddings, Stopword lists, code, and baselines for COLING 2020 paper titled "KINNEWS and KIRNEWS: Benchmarking Cross-Lingual Text Classification for Kinyarwanda and Kirundi" by Rubungo Andre Niyongabo, Hong Qu, Julia Kreutzer, and Li Huang. # KINNEWS-and-KIRNEWS Data, Embeddings, Stopword lists, code, and baselines for COLING 2020 paper titled "KINNEWS and KIRNEWS: Benchmarking Cross-Lingual Text Classification for Kinyarwanda and Kirundi" by Rubungo Andre Niyongabo, Hong Qu, Julia Kreutzer, and Li Huang. This paper introduces Kinyarwanda and Kirundi news classification datasets (KINNEWS and KIRNEWS,respectively), which were both collected from Rwanda and Burundi news websites and newspapers, for low-resource monolingual and cross-lingual multiclass classification tasks. Along with the datasets, we provide statistics, guidelines for preprocessing, pretrained word embeddings, and monolingual and cross-lingual baseline models. **Note:** Please, when using any of the resources provided here, remember to cite our paper. ## Data ### Download the datasets - The raw and cleaned versions of KINNEWS can be downloaded from here (21,268 articles, 14 classes, 45.07MB(raw) and 38.85MB(cleaned)) - The raw and cleaned versions of KIRNEWS can be downloaded from here (4,612 articles, 12 classes, 9.31MB(raw) and 7.77MB(cleaned)) ### Datasets description Each dataset is in camma-separated-value (csv) format, with columns that are described bellow (Note that in the cleaned versions we only remain with 'label','title', and 'content' columns): | Field | Description | | ----- | ----------- | | label | Numerical labels that range from 1 to 14 | | en_label | English labels | | kin_label | Kinyarwanda labels | | kir_label | Kirundi labels | | url | The link to the news source | | title | The title of the news article | | content | The full content of the news article | ## Word embeddings ### Download pre-trained word embeddings - The Kinyarwanda embeddings can be downloaded form here (59.88MB for 100d and 29.94MB for 50d) - The Kirundi embeddings can be downloaded from here (17.98MB for 100d and 8.96MB for 50d) ### Training your own embeddings To train you own word vectors, check out code/embeddings/word2vec_training.py f …