adedejimakinde/Zero-Shot-vs.-Fine-Tuned-Approaches-for-Yoruba-Sentiment-Analysis
Domain:
natural language processingRecord type:
paper
Creator:
ade
Host:
Zero-Shot vs. Fine-Tuned Approaches for Yoruba Sentiment Analysis: Examining the Role of Sub-word Tokenization and Orthographic Normalization"
# Zero-Shot vs. Fine-Tuned Approaches for Yoruba Sentiment Analysis
**Examining the Role of Sub-word Tokenization and Orthographic Normalization**
A research project comparing zero-shot large language model prompting against fine-tuned transformer models for Yoruba sentiment classification, with a focus on how diacritic (tone mark) presence affects subword tokenization and downstream performance.
---
## Overview
Yorùbá is a tonal language whose standard orthography relies on diacritical marks (tone marks and underdots) that are frequently omitted in informal digital text. This project investigates two questions:
1. How does a current-generation zero-shot large language model (Gemini) compare against fine-tuned transformer models (AfriBERTa, AfroXLMR, mBERT) on Yorùbá sentiment classification?
2. Does removing diacritics affect subword tokenization and downstream sentiment classification performance?
A full manual error analysis further examines why each approach fails, categorizing misclassified examples by linguistic content (proverbs, news reporting, political framing, social media banter, and more).
## Key Findings
- **Zero-shot outperforms all fine-tuned models** on this task, across both diacritized and undiacritized text.
- **Diacritic removal has minimal effect on classification performance** (F1 differences of −0.006 to +0.012 across all four approaches), despite diacritics measurably increasing subword tokenizer fragmentation.
- **Negative sentiment is the hardest class** for every approach tested.
- **Zero-shot and fine-tuned models fail differently**: zero-shot errors concentrate in proverbs and culturally-embedded implicit sentiment; fine-tuned model errors concentrate in informal, code-mixed social media text.
## Repository Structure
```
data/
├── combined_train.csv # Cleaned, merged dataset (NaijaSenti + YOSM), with diacritized/undiacritized text
├── split_train.csv # Training split (stratified, 80%)
├── split_val.cs …
Similar
pmmlv2-fine-tuned-yorubaCode-switched Training vs Multilingual Fine-tuning for Zero-shot Cross-lingual RetrievalCross-lingual Relation Extraction with Large Language Models: Zero-Shot, Few-Shot, and Fine-Tuned Evaluation on RomanianHybrid Batch Training vs. Domain-Adaptive Fine-Tuning for Zero-Shot Cross-Lingual Retrieval on XQuADoyemade/seamless-m4t-medium-yoruba-fine-tunedDetecting AI-Generated Paraphrases in Bengali: A Comparative Study of Zero-Shot and Fine-Tuned Transformers