bkekgathetse/setswana-offensive-977

A repository for my work on Setswana Offensive Language Detection. We are using Transformer architectures and state of the art Natural Language Processing to develop a system that is able to detect offensive content, coupled with Explainable-Artificial Intelligence (XAI) to support digital forensic investigation. # A Forensic Linguistic Dataset for Offensive Content Detection in Low-Resource Language: Setswana - Corpus: 977 items (477 offensive; 500 non-offensive) - Splits: 80/20 holdout (tag-free), 5-fold CV on 80% - Models: PuoBERTa (no triggers), Afro-XLM-R (no triggers) - Docs: see data_statement/DATASTATEMENT.md # Abstract This paper presents a monolingual Setswana dataset for offensive language detection, developed for digital forensic applications. The dataset comprises texts drawn from social media, civic discourse, and lexicographic sources, and includes span-level semantic trigger annotations. Double coding yielded strong inter-annotator agreement, with a Cohen’s kappa of 0.86. The corpus was divided in an 80:20 ratio into a training set and an untouched, semantic-trigger-free (tag-free) holdout set. Model selection was conducted using five-fold stratified cross-validation on the training split, followed by final evaluation on the holdout set. Classical machine-learning baselines, including TF--IDF with Naïve Bayes, Logistic Regression, and a Linear Support Vector Machine, were benchmarked against transformer models under tag-free training. On the tag-free holdout set, the classical models achieved macro F1 Scores of 0.80-0.87, while the transformers performed only marginally better. However, the 95\% bootstrap confidence intervals for macro F1 overlapped, and the paired Wilcoxon signed-rank test was non-significant ($p = 0.625$), with negligible effect sizes, indicating no statistically reliable difference at this corpus scale. In contrast, training with semantic trigger tags and testing on tag-free data reduced transformer performance, suggesting supervision leakage. For forensic traceability, Local Interpretable Model-agnostic Explanations (LIME) and Stabilized LIME (S-LIME) were applied. Their attributions aligned with annotated trigger spans and remained stable across perturbations, while counterfactual edits identified the conditions under which predicti …