Elsiekoech/mbert-codeswitch-classifier

Language classifier routing Swahili/English/code-switched customer messages for banking automation # Swahili / English / Code-Switched Sentence Classification (mBERT) A sentence-level language classifier for the Kenyan banking context. It labels a customer message as **Swahili**, **English**, or **Code-Switched** (a mix of both, e.g. *"Nataka ku-transfer 5000 shillings"*), so an automated system — a chatbot or routing layer — can send each query to the right downstream handler. Standard NLP systems are trained on monolingual or general multilingual data and handle mixed-language input poorly. This project shows that **fine-tuning multilingual BERT (mBERT) on a small, locally-built code-switched dataset** turns a model that is effectively guessing into an accurate classifier. ## Result | Model | Accuracy | Macro F1 | |---|---|---| | Zero-shot mBERT (no fine-tuning) | 0.33 | 0.17 | | **Fine-tuned mBERT** | **0.91** | **0.91** | Fine-tuning lifts accuracy from chance level (~33% on a balanced 3-class task) to **91%**. Most remaining errors fall between **Swahili** and **Code-Switched** — the expected confusion, since those classes share the most linguistic overlap. ## Approach 1. **Baseline (`basemodel.py`)** — evaluate pre-trained mBERT with an untrained classification head to establish a floor. With random head weights the model has no task knowledge, so its ~33% sets the bar to beat. 2. **Fine-tuning (`finetunedmodel.py`)** — fine-tune mBERT on the training set (3 epochs, AdamW, linear warm-up, gradient clipping) and evaluate with accuracy, F1, precision/recall, and a confusion matrix. ## Dataset A **locally created**, synthetically generated and manually curated dataset reflecting code-switching patterns common in Kenyan banking queries: - Training: 237 sentences · Test: 210 sentences - Three balanced classes: `English`, `Swahili`, `Code-Switched` The two files use slightly different label spellings (`Code-Switched` vs `code_switch`); both scripts normalise labels to one canonical mapping (`english=0, swahili=1, code-switched=2`) before training. **T …