ReneNtabana/African-Language-Health-QA-Challenge

# Multilingual Health Question Answering in Low-Resource African Languages Final project for the **Multilingual Health Question Answering in Low-Resource African Languages Challenge** (Zindi, co-organized by ITU, HASH, and Makerere University Centre for AI). ## Overview This project builds a question-answering system for maternal, sexual, and reproductive health questions across English, Akan, Luganda, Swahili, and Amharic. Rather than fine-tuning a generative model from scratch, the core approach is **retrieval-based**: a per-language hybrid index combines TF-IDF (lexical) and sentence-embedding (semantic) similarity to find the most similar previously-answered question for any new query, then returns that answer. A supplementary experiment fine-tunes `google/mt5-small` as a generative alternative, to compare retrieval against generation under the same evaluation protocol. **Why retrieval first:** with a training set of real, pre-written, expert-style health answers, copying the answer to the most similar existing question is a strong, fast, low-risk baseline — before paying the cost (and instability risk) of fine-tuning a multilingual generative model on a relatively small dataset. ## Results Summary | Experiment | Change | Val ROUGE-1 | Val ROUGE-L | |---|---|---|---| | E01 | TF-IDF only baseline | 0.393 | 0.336 | | E02 | Semantic-only retrieval | 0.413 | 0.361 | | E03 | Fixed-weight hybrid (0.35 TF-IDF / 0.65 semantic) | 0.433 | 0.377 | | E04 | Per-subset tuned hybrid weights | 0.460 | 0.407 | | E05 | Exact-match lookup + tuned fallback | 0.460 | 0.407 | | E06 | Deduplicated training corpus | 0.460 | 0.407 | | **E07** | **Char-level TF-IDF for low-resource subsets** | **0.461** | **0.408** | | E08 | Cross-subset fallback for low-resource languages | 0.459 | 0.406 | | E09 | mT5-small fine-tune (2 epochs, 25% data) | 0.080 | 0.071 | | E10 | Final comparison — selects best config | — | — | | E12 | Encoder swap to `multilingual-e5-large` | ~0.49 | ~0.44 | …