Francklin9999/low-resource-qa-representation

Benchmarking multilingual QA in low-resource African languages: direct vs translation pipelines and robustness to representation gaps. # Multilingual QA Bench **Benchmarking Large Language Models on Extractive Question Answering for Low-Resource African Languages** 📄 **Read the paper (PDF)** — full methodology, results, and analysis. ## Motivation Large language models achieve remarkable performance on English NLP tasks, yet their capabilities on low-resource languages remain understudied. For many African languages, there is an open question: should we query LLMs *directly* in the target language, or *translate* to English first, query the LLM, and translate the answer back? This project provides an empirical answer by systematically benchmarking both strategies across 10 African languages, 5 model families, and a locally fine-tuned model. ## Research Questions 1. **Direct vs. translate-pivot**: For a given LLM, does querying directly in the source language outperform translating to English and back? 2. **Model scale vs. specialization**: Can a small, fine-tuned model (Qwen2.5-3B with QLoRA) compete with frontier API models (GPT-5.3, Claude Opus 4.5, Gemini 2.5 Pro)? 3. **Language resource level**: How does performance vary across languages with different levels of representation in pretraining data? ## Experimental Setup ### Dataset We use AfriQA (Ogundepo et al., 2023), a cross-lingual open-retrieval QA dataset for 10 African languages. Each example contains a question, a gold Wikipedia passage (context), and a reference answer in the source language. ### Languages | Code | Language | Family | Pivot | Test Samples | |------|-------------|----------------|-------|:------------:| | hau | Hausa | Afro-Asiatic | en | 45 | | yor | Yoruba | Niger-Congo | en | 51 | | ibo | Igbo | Niger-Congo | en | 62 | | swa | Swahili | Niger-Congo | en | -- | | kin | Kinyarwanda | Niger-Congo | en | 44 | | twi | Twi | Niger-Congo | en | 71 | | wol | Wolof …