AmanuelZ/amharic-nlp-exploration

Exploring the state of NLP for Amharic — tokenizer analysis, dataset survey, model evaluation, and sentiment fine-tuning # Amharic NLP Exploration (አማርኛ) Exploring the current state of NLP for Amharic — Ethiopia's official language with 50+ million speakers — through the lens of a native speaker and software engineer. ## Why This Exists Amharic is one of the most widely spoken languages in Africa, yet it remains severely underrepresented in NLP research and commercial language models. Standard tokenizers fragment Amharic words into meaningless sub-tokens, existing benchmarks barely cover Ethiopic script languages, and most "multilingual" models perform poorly on Amharic tasks. This project provides: 1. **Tokenizer analysis** — quantifying the "tokenization tax" Amharic speakers pay 2. **Dataset landscape** — what exists, what's missing, where to invest 3. **Model evaluation** — testing existing models with native speaker assessment 4. **Sentiment fine-tuning** — end-to-end fine-tuning of XLM-RoBERTa on Amharic sentiment 5. **Native speaker annotations** — linguistic insights that non-speakers cannot provide ## Key Findings ### Tokenization Tax Amharic text requires 2-13x more tokens than equivalent English text across all major multilingual tokenizers: | Tokenizer | Amharic Fertility | English Fertility | Ratio | |-----------|------------------|-------------------|-------| | GPT-2 | 14.4 tokens/word | 1.14 tokens/word | **12.63x** | | mBERT | 1.04 (all UNK) | 1.17 | 0.89x | | XLM-RoBERTa | 2.62 | 1.20 | **2.18x** | | NLLB-200 | 2.62 | 1.18 | **2.22x** | | mT5 | 3.88 | 1.27 | **3.06x** | This means higher API costs, shorter effective context windows, and degraded model performance for Amharic speakers. ### Dataset Gaps - No conversational Amharic corpus - No instruction-following dataset (critical for LLM alignment) - Existing corpora heavily biased toward religious and news text - No benchmarks for code-switching (Amharic-English mixing) ### Model Performance - **mBERT**: Returns `[UNK]` for all Amharic tokens — zero Amharic vocabulary - **XLM-RoBERTa**: Predicted "South" in …