Omballa/dholuo-pos-tagger-finetuned

# 🇰🇪 Dholuo POS Tagger — AfroXLM-R (KenCorpus Fine-tuning) This repository contains the **training code, preprocessing pipeline, and documentation** for a **Dholuo (Luo) Part-of-Speech (POS) tagging model**, fine-tuned from **AfroXLM-R** using **KenCorpus** data. The final trained model is published on Hugging Face: 👉 **Hugging Face Model:** Omballa/dholuo-pos-afroxlmr… This work follows **Masakhane-aligned practices** for low-resource African NLP. --- ## ✨ Overview - **Task:** POS tagging (token classification) - **Language:** Dholuo (Luo) - **Base Model:** `masakhane/luo-pos-tagger-afroxlmr` - **Architecture:** AfroXLM-R - **Training Data:** KenCorpus (Dholuo) - **Framework:** Hugging Face Transformers - **Output:** UPOS / Masakhane-aligned POS tags --- ## 📂 Repository Structure ```text . ├── data/ │ ├── data_upos.csv │ └── README.md # Detailed data documentation & preprocessing ├── training/ │ ├── train.py # Training script │ ├── evaluate.py # Evaluation utilities │ └── utils.py ├── notebooks/ │ └── exploration.ipynb ├── README.md # This file └── LICENSE ```` > 📌 **Important:** > See `data/README.md` for full details on **KenCorpus cleaning**, **POS tag alignment**, and **data limitations**. --- ## 📚 Dataset ### KenCorpus (Dholuo) The training data originates from **KenCorpus**, a Kenyan language corpus designed to support NLP research for low-resource languages. * **Language:** Dholuo * **Annotations:** POS tags * **Characteristics:** * Fine-grained and composite POS tags * Corpus-specific labels * Token-level annotation To ensure compatibility with transformer-based models and Masakhane tooling, the data was **cleaned and normalized** into a **Masakhane / Universal POS (UPOS)** label set. 📄 Full details are documented in: ➡️ `data/README.md` --- ## 🧹 Preprocessing Summary The preprocessing pipeline includes: * Token and tag normalization * Removal of malfor …