Michael-Beukman/NERTransfer

Investigating transfer learning in low-resourced languages, specifically in a named entity recognition (NER) task (IJCNLP-AACL 2023). Analysing Cross-Lingual Tra… # NER Transfer - Interpretation and Analysis # About This is an project that investigates transfer learning in low-resourced languages, specifically in a named entity recognition (NER) task. This repository contains the code, the trained models can be found here and the paper (accepted to IJCNLP-AACL 2023) can be found here. Further, many visualisations are available in ./analysis/. Finally, the raw predictions and results can be found in ./src/runs/v10/models/ # Contents - NER Transfer - Interpretation and Analysis - About - Contents - Code Structure - Get Started - Train Models - Evaluate - Analysis - Exploration - References / Sources / Libraries - License - Citation - Model Cards - About - Contact & More information - Training Resources - Data - Intended Use - Limitations - Privacy & Ethical Considerations - Metrics - Caveats and Recommendations - Model Structure - Usage # Code Structure We mainly did the following in this project: 1. Fine-tune many different pre-trained models on different languages from the MasakhaNER dataset. 2. Using these fine-tuned models, we evaluated on all of the languages, to get an idea for zero-shot potential. 3. Then we moved onto analysis, which is the bulk of our contributions and code here. The specific analysis we did can be found in `./src/analysis`, with the following subfolders: 1. `v10`: Analyse Pre-training effect on final performance. 2. `v20`: Look at zero-shot transfer. 3. `v40`: Investigate word embeddings, performing, among others, PCA on them and plotting them. 4. `v50`: Statistically analysing the data overlap between the different languages, and investigating correlation between this and performance. ``` ├── analysis -> This contains the bulk of our results, including plots, csv files and Latex tables. │   ├── v10 │   ├── v20 │   ├── v40 │   └── v50 ├── data │   └── masakhane-ner ├── doc │   ├── report.pdf -> Written Report ├── env.yml -> …