sanderlj/nlp-finetuning

NLP project on finetuning mBERT-small on Swahili NLP tasks # CSC5035Z Assignment 2 This repository contains code, configuration files, and final outputs for fine-tuning `mmBERT-small` on two Swahili NLP tasks: - `AfriSenti` for text classification - `MasakhaPOS` for token classification It also includes an Extension D comparison of parameter-efficient fine-tuning (PEFT) methods on the AfriSenti task. ## Project Structure - `src/` - training code and helper functions - `scripts/` - command-line entrypoints for running experiments - `configs/` - YAML configuration files for baseline, tuning, and extension runs - `results/` - saved metrics and error-analysis files for final runs - `slides/` - presentation file - `report/` - report file ## Setup Create and activate a virtual environment, then install dependencies: ```bash python -m venv .venv ``` Windows PowerShell: ```bash .\.venv\Scripts\activate ``` Install requirements: ```bash pip install -r requirements.txt ``` ## Running Experiments ### Final baseline text classification ```bash python scripts/train_text.py --config configs/text_final.yaml ``` ### Final baseline token classification ```bash python scripts/train_token.py --config configs/token_final.yaml ``` ### Text tuning runs ```bash python scripts/train_text.py --config configs/text_tune_ep3_lr2.yaml python scripts/train_text.py --config configs/text_tune_ep3_lr3.yaml python scripts/train_text.py --config configs/text_tune_ep5_lr2.yaml python scripts/train_text.py --config configs/text_tune_ep5_lr3.yaml ``` ### Token tuning runs ```bash python scripts/train_token.py --config configs/token_tune_ep3_lr2.yaml python scripts/train_token.py --config configs/token_tune_ep3_lr3.yaml python scripts/train_token.py --config configs/token_tune_ep5_lr2.yaml python scripts/train_token.py --config configs/token_tune_ep5_lr3.yaml ``` ### Extension D: PEFT comparison on AfriSenti ```bash python scripts/train_text.py --config configs/text_lora_r4.yaml python scripts/train_text.py --config configs/text_lora_r8.ya …