OmondiKevin/afriqa-entity-aware-qa

Entity-aware adaptation for direct African-language XOR-QA on AfriQA (no translation), using MasakhaNER supervision and EM/F1 + semantic similarity evaluation. # Training-Schedule Sensitivity in African Cross-Lingual QA This repository contains the code and reproducibility workflows for African cross-lingual gold-passage question answering on AfriQA. Papers, reference PDFs, generated evidence, checkpoints, and local review builds are intentionally not versioned. The task combines a Hausa, Yoruba, or Swahili question with an English evidence passage and requires an answer in the question language. Hausa and Yoruba occur in the 470-example training set; Swahili is evaluated zero-shot. ## Confirmed result Across paired seeds 13, 42, and 101: | System | Exact match | Token F1 | |---|---:|---:| | Direct mT5, original-data schedule | 3.54 ± 0.24 | 7.43 ± 0.10 | | Direct mT5, 20×-upsampled schedule | 16.16 ± 3.77 | 19.45 ± 4.15 | | Corrected translation + reader | **32.67** | **45.56** | The upsampled direct condition improves in all three seeds, including on zero-shot Swahili. It does not beat the corrected translation pipeline overall. The “20×” label describes dataset duplication, not actual selected optimizer updates. Selected checkpoints use only 1.25–1.56× as many updates as the original-data checkpoints and follow a different learning-rate trajectory. A predeclared five-seed fixed-endpoint study now separates two components of that schedule: | Controlled contrast | ΔEM (95% interval) | ΔF1 (95% interval) | Positive seeds | |---|---:|---:|---:| | Long vs. short scheduler horizon at 160 updates | +2.92 (1.77–4.20) | +2.81 (1.61–4.11) | 5/5 | | 250 vs. 160 updates under the same long horizon | +9.91 (6.65–12.88) | +10.38 (6.97–13.29) | 5/5 | Both scheduler position and update budget matter, but the extra 90 updates explain most of the observed separation in this range. Repetition itself is held constant across the controlled conditions, so these experiments do not establish a causal benefit of materializing duplicate records. ## Repository structure - `submission_experiments_colab.ipynb` — exact clean-run workflow …