kaglet/afrikaans_sem_rel

# Evaluating Cross-Lingual Semantic Relatedness in African Languages This project investigates the effectiveness of multilingual pre-trained language models for measuring semantic relatedness in low-resource African languages, with a focus on Afrikaans. We benchmark traditional statistical approaches and embedding-based baselines against fine-tuned transformer models like AfroXLM-R and LaBSE. The study also utilizes interpretability methods, including LIME, to examine model reasoning. ## Run Instructions 1. **Fork and Clone the Repository:** ```bash git clone github.com cd your-repository ``` 2. In root directory run ```bash pip install -r requirements.txt ``` 3. **Run Baselines (Training and Metrics Output):** ```bash python baselines/cls_embeddings_baseline.py python baselines/tfidf_baseline.py ``` 4. **Run Fine-Tuned Models (Training and Metrics Output):** * Open and run the Jupyter notebooks: ```bash jupyter notebook finetune_models/finetuning_afroxlmr.ipynb jupyter notebook finetune_models/finetuning_labse.ipynb ``` ## Problem Statement * Which multilingual pre-trained language models (AfroXLM-R, LaBSE) achieve the best performance in identifying semantic relatedness in African language texts when fine-tuned using transfer learning? * Can interpretability tools such as LIME help explain how these models arrive at their predictions? ## Datasets The project uses a combination of: * **SemRel2024 (Afrikaans Subset):** 751 human-annotated sentence pairs from SemEval-2024 Task 1. * **SemRel2022 (English):** 5499 English sentence pairs, back-translated into Afrikaans using Google Translate for data augmentation. The combined dataset is split into 70% for training and 30% for testing using stratified sampling. ## Models ### Baseline Models * **TF-IDF to Linear Regression:** A traditional statistical approach. * **CLS Embeddings to Linear Regression:** Utilizes frozen AfroXLMR embeddings. ### Fine-Tuned Models …