rulezcasa/Cross-lingual-Kinyarwanda_Kirundi

Benchmarking cross lingual transfer on various Transformer and traditional Neural Models. # Cross-lingual transfer of multilingual models on low resource African Languages ## Overview Benchmarking cross-lingual transfer from Kinyarwanda to Kirundi using transformer(mBERT, AfriBERT, BantuBERTa) and traditional neural models (BiGRU, CNN, char-CNN). While monolingual models remain competitive, the analysis highlights the strong cross-lingual transfer capabilities in resource limited settings of transformer multilingual architectures. ## Pre-print manuscript Cross-lingual transfer of m… ## Directory structure ``` project_root/ ├── requirements.txt ├── embeddings.ipynb ├── Transformer_Architectures/ │ ├── mBERT.ipynb │ ├── AfriBERT.ipynb │ └── BantuBERT.ipynb │ ├── Traditional_Architectures/ │ ├── BiGRU.ipynb │ ├── CNN.ipynb │ └── CharCNN.ipynb │ └── Dataset_Cleaned/ ├── zero_kin_train.csv ├── zero_kin_test.csv ├── zero_kir_train.csv └── zero_kir_test.csv ``` ## Dataset The dataset comprises of news articles from both languages Kinyarwanda and Kirundi labelled with the following classes sourced from 10.18653/v1/2020.coling-main.480. | Index | Category | |-------|----------------| | 1 | Politics | | 2 | Sport | | 3 | Economy | | 4 | Health | | 5 | Entertainment | | 6 | History | | 7 | Technology | | 8 | Tourism | | 9 | Culture | | 10 | Fashion | | 11 | Religion | | 12 | Environment | | 13 | Education | | 14 | Relationship | The corpus was subjected to cleaning and converting the numberical labels to zero based. The cleaned comma seperated value dataset can be found on under ```Dataset cleaned``` directory of this repository. ## Results (a) Performance before and after fine tuning | Model | Accuracy before FT | F1 before FT | Accuracy after FT | F1 after FT | |-------------|--------------------|--------------|-------------------|-------------| | mBERT | 0.5872 …