sweetlhare/itu-afri-health-qa

Zindi solution — multilingual maternal/reproductive health QA across 8 low-resource African languages (public LB 0.70) # ITU — Multilingual Health QA in Low-Resource African Languages Solution code for the Zindi competition **Multilingual Health Question Answering in Low-Resource African Languages** (ITU / HASH). Given a maternal / sexual / reproductive-health question in one of eight `language × country` subsets, the system produces an answer in the **same language**. The final approach (**v18**) reaches **public LB 0.700511**. ``` LB = 0.37 · ROUGE-1 F1 + 0.37 · ROUGE-L F1 + 0.26 · LLM-Judge ``` ROUGE is computed with whitespace tokenization and **no stemmer** — see `scripts/score.py` for an exact offline reimplementation of the ROUGE part. --- ## Key insight: the task splits into two regimes The 8 subsets behave very differently because their reference answers come from different deployed chatbots: | Regime | Subsets (test rows) | Nature | Strategy | |---|---|---|---| | **Pool / retrieval** | Eng_Uga (744), Lug_Uga (374), Swa_Ken (229), Eng_Ken (167), Eng_Eth (60) | answers reuse a closed bank of canonical FAQ replies (high paraphrase duplication) | **retrieve** the closest pooled answer | | **Generative** | Aka_Gha (492), Eng_Gha (491), Amh_Eth (61) | ~98–99% unique answers (LLM-generated chatbots) — nothing to retrieve | **generate** with a fine-tuned LLM | So ~60% of the test set is really a paraphrase-**matching** problem (a verbatim correct answer scores ROUGE ≈ 1.0), and ~40% is genuine generation. v18 is a per-subset **router** that sends each subset to its best-performing component. ## The v18 router `scripts/build_v18.py` assembles one prediction per row, choosing the source by subset: | Subset(s) | Source | Model | Post-processing | |---|---|---|---| | Eng_Uga, Lug_Uga, Swa_Ken, Eng_Ken, Eng_Eth | embedding-soup retrieval (top-1) | `bge-m3-ft-tv` + `afroxlmr-ft-tv` | — | | Aka_Gha | generation, **beam=4, length_penalty=0.7** | `afrique-gha` (AfriqueGemma-12B QLoRA) | `clean` + `snap_trim(120)` | | Eng_Gha | generation, **RAFT** (k=3 retrieved Q-A context …