A Domain-specific Large Language Model for Diabetes Care and Management

Abstract The prevalence of diabetes is rapidly increasing in low- and middle-income countries (LMIC), making it one of the fastest-growing global health emergencies of the modern era. Despite efforts by healthcare practitioners, the government, and communities to minimise the associated complications and mortality, there are significant challenges that can be potentially alleviated through innovative digital health technologies. This study developed a domain-specific large language model (DS-LLM) aimed at improving diabetes care and management using a case study of South Africa. To achieve this, local data was collected and supplemented with benchmark and medical Hugging Face datasets. Medical pre-trained large language models (LLMs): BioMedLM (2.7B) and BioMistral-7B were selected as base models, along with Qwen3-8B (a non-specialised LLM). Two Parameter-Efficient Fine-Tuning (PEFT) techniques: prompt tuning and Quantised Low-Rank Adaptation (QLoRA) were applied, with Retrieval-Augmented Generation (RAG) applied on the best-performing LLM. The fine-tuned LLMs were evaluated by comparing their respective performance with Diabetica-7B , a specialised diabetes LLM. The final dataset comprised 18,079 processed question-answer pairs (14% artificially generated for the South African context) and 1,596 documents, covering medication, management, diagnosis, screening, and general diabetes topics that pertain to South Africa. For fill-in-the-blank and multiple-choice questions formats, Qwen QLoRA outperformed all LLMs (ROUGE-1 = 0.793, ROUGE-L= 0.792, and BERTScore F1 = 0.940). Diabetica had the highest BLEU score (0.465), while Qwen3-8B had 0.365. For multiple-choice questions only, Qwen3-8B QLoRA achieved a top accuracy of 80.7%. For short and long answers, BioMistral-7B QLoRA performed slightly better, with all models scoring above 0.800. These findings highlight the promising use of LLMs for diabetes care.