NaomiMeseret/amharic-qa-data-efficient-llm

# Data-Efficient Instruction Tuning for Amharic Question Answering **Naomi Meseret** Research Assignment: Data and Compute-Efficient Generative AI — August 2026 ## Abstract Training large language models usually requires large datasets and powerful GPUs, which is difficult for low-resource languages such as Amharic. This project studies an Amharic question-answering task and investigates how the performance of an instruction-tuned LLM changes when trained on 100%, 25%, and 10% of the available QA instructions. We fine-tune the Walia-I Amharic LLaMA checkpoint with LoRA and measure both ROUGE-L and training time on a single T4 GPU. Performance does not scale monotonically with data size: the 25% condition outperforms both the full 100% dataset and the untrained starting checkpoint, while the 10% condition underperforms that same checkpoint — indicating that more fine-tuning data is not always better under a fixed training budget. ## Research question > How does the performance of an Amharic question-answering model change when trained > with 100%, 25%, and 10% of the available QA instruction data, and what is the > corresponding trade-off in GPU training time on a single T4? ## Hypothesis We expected the 25% condition to achieve performance close to the full 100% dataset while using substantially less GPU time, and the 10% condition to fall noticeably short of both. In other words, we expected most of the performance gains from instruction tuning to come from a moderate subset of the data, with additional data beyond that point yielding diminishing returns relative to its extra compute cost. **The result exceeded this hypothesis**: 25% did not just match 100% — it outperformed it outright, and also outperformed the untrained starting checkpoint. 10% fell short not just of 100%, but of the checkpoint itself. ## Research gap | Prior work | Focus | What's missing | |---|---|---| | Walia-LLM (Azime et al.) | Task and data *type* mixing for Amharic instruction t …