Evaluating Large Language Models for Low-Resource Multilingual Machine Translation in the Medical Domain

This dissertation explores neural machine translation (NMT) in multilingual medical domain, with a focus on clinical low-resource settings. Conducted in the context of the BabelDr and PictoDr projects, which aim to support communication between patients and healthcare providers, this work addresses the translation of medical utterances across 13 language combinations, including several low-resource languages such as Tigrinya, Dari, and Tunisian Arabic. It also investigates translation into biomedical glosses (UMLS) and pictogram representations to support non-verbal communication.

We evaluate domain-adapted NMT systems trained from scratch using in-domain data, and compare them with large language models used in zero-shot, fewshot settings and fine-tuning. A multilingual evaluation framework is proposed, combining automatic metrics with expert-based human assessment focusing on adequacy, fluency, and usability in clinical scenarios. Our findings show that while general-purpose LLMs demonstrate promising capabilities, they fall short on domain-specific tasks in low-resource languages. In contrast, domain-specific NMT systems offer more reliable outputs.

This work highlights the importance of domain adaptation, evaluation, and inclusive language coverage for translation tools in multilingual healthcare environments.