yakobd/gheero-amharic-efficient-summarization

Research on quality- and diversity-based data selection for compute-efficient Amharic summarization with mT5-small — Gheero AI/ML Residency individual research project. # Amharic Efficient Summarization Data-efficient fine-tuning of mT5-small for Amharic abstractive summarization, via quality-and-diversity-based training data selection. ## Research question Can a quality-and-diversity-based data selection method achieve summarization performance comparable to full-dataset fine-tuning, using a much smaller subset, for adapting mT5-small to Amharic summarization — and does it meaningfully beat random subset selection at the same budget? ## Hypothesis A subset chosen to be both high-quality (learnable, well-aligned article/summary pairs) and diverse (spread across topical clusters) will recover most of the ROUGE performance of full-dataset fine-tuning at a fraction of the training data and compute, and will outperform a same-size random subset. ## Method: Quality-Filtered Diverse Sampling 1. **Q2 — quality filter**: drop low-fidelity article/summary pairs using an overlap/compression signal between article and summary text. 2. **Q1 — quality score**: score remaining pairs by the loss a **pretrained** (not fine-tuned) mT5-small assigns to the reference summary — a proxy for how learnable each example is. 3. **D1 — diversity**: embed articles with LaBSE (`sentence-transformers/LaBSE`), cluster with k-means, and sample proportionally across clusters, prioritizing higher Q1+Q2 examples within each cluster. ## Dataset XL-Sum, Amharic config (`csebuetnlp/xlsum`, `amharic`) — ~5,461 article-summary pairs, using XL-Sum's own predefined train/validation/test splits. ## Model `google/mt5-small` ## Experiment matrix | # | Run name | Subset size | Selection method | |---|------------------------|-------------|-------------------------| | 1 | `full_dataset` | 100% | none (ceiling baseline) | | 2 | `random_25pct` | 25% | random | | 3 | `quality_only_25pct` | 25% | Q1+Q2 quality only | | 4 | `diversity_only_25pct` | 25% | D1 diversity only | …