Advancing Tigrinya Multiclass Text Classification with Pre-trained Models: A Comparative Analysis

Pre-trained language models have driven remarkable progress in Natural Language Processing (NLP). However, these advances have favored high-resource languages, leaving low-resource languages like Tigrinya, spoken by over 10 million people, underrepresented. This study comprehensively evaluates several pre-trained language models for multiclass text classification on a newly curated Tigrinya dataset spanning seven topical categories. We investigate the Performance of monolingual models (TiBERT, TiRoBERTa, TiELECTRA), multilingual models (mBERT, XLM-R, Afro-XLM-R, SERENGETI), and generative models (GPT-2, GPT-2-Tigrinya) under various tokenization and preprocessing conditions. Results show that monolingual models fine-tuned on Tigrinya consistently outperform their multilingual counterparts, with TiRoBERTa achieving the highest macro-F1 score of 0.9023. Notably, SERENGETI, a multilingual model, surpasses TiBERT, demonstrating the promise of regionally optimized multilingual pretraining. We further demonstrate that SentencePiece consistently yields the best Performance among the five evaluated custom tokenization strategies, significantly enhancing the effectiveness of mBERT and GPT-2 models. We also show that transliterating text from the Ge'ez script to Latin improves Performance for models unfamiliar with the native script. However, this comes at the cost of losing specific script-specific nuances. Our findings underscore the importance of languagespecific pretraining, tokenizer adaptation, and script-aware preprocessing in advancing NLP for low-resource languages. This work presents a reproducible framework for Tigrinya NLP and provides practical insights for extending methodologies to other underrepresented languages. The codebase and the newly curated dataset are accessible for research at github.com.