Elizabeth-Mwania/part-of-speech-tagging-africa

This project implements Part-of-Speech (POS) tagging system for Swahili (Kiswahili), a morphologically rich African language. By utilizing Transfer Learning from multilingual transformer models (specifically Afro-XLMR) on datasets from the Masakhane POS project. # Swahili Part-of-Speech (POS) Tagging with Transformer Architectures ## Overview This project implements Part-of-Speech (POS) tagging system for Swahili (Kiswahili), a morphologically rich African language. By utilizing Transfer Learning from multilingual transformer models (specifically Afro-XLMR) on datasets from the Masakhane POS project, enabling accurate POS tagging and token-level grammatical classification. The project supports model architectures and provides a complete pipeline from data preprocessing to model training, evaluation, and prediction. It is particularly useful for researchers and practitioners working on Natural Language Processing (NLP) tasks in African languages. ## Architecture The project utilizes a Transformer-Encoder + Token-Classification Head architecture. - Feature Engineering & Tokenization Swahili presents a unique challenge: a single word (e.g., Ninakupenda) can encode a subject, tense, object, and verb root. Subword Modeling uses SentencePiece/BPE to decompose agglutinative structures. Label Alignment implemented a masking strategy where only the initial subword of a decomposed token carries the gold-standard label, while subsequent subwords are assigned a padding index (-100) to be ignored by the CrossEntropyLoss function. - Model Selection Primary Backbone: Davlan/afro-xlmr-mini (XLM-RoBERTa optimized for African languages). Alternative Backbones: Architecture-agnostic support for BERT, RoBERTa, and DistilBERT. Rationale: XLM-R’s pre-training on 100+ languages provides a cross-lingual transfer advantage that is critical for Swahili's low-resource setting. ## Technology - Deep Learning Framework: PyTorch - Model Hub: Hugging Face Transformers - Optimization: AdamW with Linear Warmup/Decay - Evaluation: seqeval (accuracy, precision, recall, loss rate) - Telemetry: TensorBoard / Python Logging ## Methodology ### Data Pipeline The system ingests CoNLL-formatted data. The preprocessing layer transforms raw strings i …