JONNY-ME/multilingual-health-qa

Colab-based experiments for multilingual maternal, sexual, and reproductive health question answering in low-resource African languages. # Multilingual Health QA Notebooks Colab experiment tracker for multilingual health question answering in low-resource African languages. ## Goal The task is to generate accurate, fluent, and contextually appropriate health answers in the same language as the input question. The challenge focuses on low-resource African languages including Akan/Twi, Luganda, Kiswahili, and Amharic, alongside English subsets from several countries. ## Evaluation Use a weighted multi-metric score: | Metric | Weight | Purpose | | --- | ---: | --- | | ROUGE-1 F1 | 0.37 | Keyword and unigram overlap | | ROUGE-L F1 | 0.37 | Sentence-level sequence similarity | | LLM-as-a-Judge | 0.26 | Factuality, completeness, and language appropriateness | Prediction files contain these columns for evaluation: ```text ID, TargetRLF1, TargetR1F1, TargetLLM ``` ## Current Baseline Current public leaderboard experiment log: | Run | Model | Setup | ROUGE-L F1 | ROUGE-1 F1 | LLM Judge | Weighted PLB Score | | --- | --- | --- | ---: | ---: | ---: | ---: | | `gemma4-e4b-zero-shot-v1` | `google/gemma-4-E4B-it` | Zero-shot, Colab GPU | 0.3663 | 0.6071 | 0.6133 | 0.5196 | | `gemma4-26b-a4b-zero-shot-v1` | `google/gemma-4-26B-A4B-it` | Zero-shot, Colab GPU | 0.3655 | 0.6277 | 0.7720 | 0.5682 | Runtime note: the run took about 3 hours and used roughly 15 GB RAM in the Colab environment. ## Experiment Roadmap 1. Establish zero-shot Gemma 4 E4B-it baseline. 2. Try larger Gemma 4 variants, starting with `google/gemma-4-26B-A4B-it`. 3. Compare prompt variants on stratified validation samples before full test generation. 4. Add retrieval-based baselines to improve ROUGE-style lexical overlap. 5. Move to QLoRA fine-tuning only after baseline and retrieval runs are well understood. ## Main Notebook This repository is intended to upload notebooks only, with notebooks placed at the repository root. The current baseline notebook is: ```text 01_gemma4_e4b_zero_shot_baseline.ipynb ``` It supports: - Colab GP …