Development and Evaluation of HEAL A Bilingual English-Luganda RAG Chatbot for Disease Surveillance in Uganda

Abstract Frontline health workers in Uganda are often unable to use disease surveillance guidelines effectively: the Pdf documents are long, technical, and written in English, leaving Luganda-speaking workers without practical access. Off-the-shelf large language models generate plausible but ungrounded advice that can contradict national protocols. We developed HEAL, a bilingual chatbot using retrieval-augmented generation to deliver Uganda's Integrated Disease Surveillance and Response guidelines in English and Luganda, built on GPT-4-turbo with a fine-tuned MBart translation model trained on parallel English-Luganda sentence pairs. Six disease surveillance experts rated HEAL against three commercial models on field-generated questions across relevance, coverage, and coherence, while twenty frontline health workers gave qualitative feedback. HEAL scored higher than all baseline models in pooled analysis, with reviewers rating it highest most consistently. Grounding a large language model in national guidelines alongside a local-language translation layer produced higher expert ratings than commercial alternatives, an approach adaptable to other languages and guidelines.