adedejimakinde/Zero-Shot-vs.-Fine-Tuned-Approaches-for-Yoruba-Sentiment-Analysis

Zero-Shot vs. Fine-Tuned Approaches for Yoruba Sentiment Analysis: Examining the Role of Sub-word Tokenization and Orthographic Normalization" # Zero-Shot vs. Fine-Tuned Approaches for Yoruba Sentiment Analysis **Examining the Role of Sub-word Tokenization and Orthographic Normalization** A research project comparing zero-shot large language model prompting against fine-tuned transformer models for Yoruba sentiment classification, with a focus on how diacritic (tone mark) presence affects subword tokenization and downstream performance. --- ## Overview Yorùbá is a tonal language whose standard orthography relies on diacritical marks (tone marks and underdots) that are frequently omitted in informal digital text. This project investigates two questions: 1. How does a current-generation zero-shot large language model (Gemini) compare against fine-tuned transformer models (AfriBERTa, AfroXLMR, mBERT) on Yorùbá sentiment classification? 2. Does removing diacritics affect subword tokenization and downstream sentiment classification performance? A full manual error analysis further examines why each approach fails, categorizing misclassified examples by linguistic content (proverbs, news reporting, political framing, social media banter, and more). ## Key Findings - **Zero-shot outperforms all fine-tuned models** on this task, across both diacritized and undiacritized text. - **Diacritic removal has minimal effect on classification performance** (F1 differences of −0.006 to +0.012 across all four approaches), despite diacritics measurably increasing subword tokenizer fragmentation. - **Negative sentiment is the hardest class** for every approach tested. - **Zero-shot and fine-tuned models fail differently**: zero-shot errors concentrate in proverbs and culturally-embedded implicit sentiment; fine-tuned model errors concentrate in informal, code-mixed social media text. ## Repository Structure ``` data/ ├── combined_train.csv # Cleaned, merged dataset (NaijaSenti + YOSM), with diacritized/undiacritized text ├── split_train.csv # Training split (stratified, 80%) ├── split_val.cs …