Logo Lanfrica
  • Accueil
  • Atlas
  • Analyses
  • Documentation
  • Sign in

© 2026 Lanfrica. Tous droits réservés. Tous les droits d'auteur des ressources affichées sur le site Web Lanfrica appartiennent aux détenteurs de droits d'auteur d'origine, sauf indication contraire explicite.

Safety That Does Not Transfer: Cross-Lingual Clinical Correctness Drift in Deployable Medical Language Models

Domaine:

natural language processinghealthcare

Type de record:

paper
Créateur:
GabOlaAjaAde
Éditeur:
arXiv
Hôte:avatar
Safety evaluation of large language models is conducted predominantly in English and predominantly on frontier systems. Neither condition describes how such models are encountered in low-resource health settings, where small quantised systems are run locally and queried in local languages. We ask whether clinical safety established in English transfers to Hausa, and whether any failure is attributable to the language, the clinical task, or the class of model that low-resource deployment admits. Matched English-Hausa question pairs were built for three conditions of high burden in northern Nigeria: malaria, sickle cell disease, and tuberculosis, probing knowledge recall, emergency triage, a leading question inviting a contraindicated action, and a traditional-remedy claim. Six models were evaluated: five locally deployable systems of 4-9 billion parameters, two medically fine-tuned, and one frontier system. All 128 responses were scored against Nigerian national treatment guidelines by two fluent Hausa speakers working independently and blind to one another. Among locally deployable models, mean clinical correctness fell from 1.57 in English to -0.03 in Hausa, on a scale where 2 denotes a correct answer and -1 an actively harmful one. The frontier model moved from 2.00 to 1.75 and produced no response judged harmful in either language. Drift was consistent across all three conditions. Inter-rater agreement was substantial for clinical correctness (kappa = 0.70); agreement on harm was initially poor (kappa = 0.22) and is examined in detail. Because a frontier model answers the same questions competently in Hausa, the deficit is a property neither of the language nor of the clinical material, but of the deployable tier.

Visit

doi.org

Languages

Hausa

Tags

Computation and Language (cs.CL)Computers and Society (cs.CY)FOS: Computer and information sciences

Licenses

Creative Commons Attribution 4.0 Internationalhttps://creativecommons.org/licenses/by/4.0/legalcode

Similaires

Analyzing the Evaluation of Cross-Lingual Knowledge Transfer in Multilingual Language ModelsBUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual TransferAnd Where that Language Does Not Yet ExistEffects of Language Relatedness for Cross-lingual Transfer Learning in Character-Based Language ModelsWhen Does Language Transfer Help? Sequential Fine-Tuning for Cross-Lingual Euphemism DetectionCode-Switching In-Context Learning for Cross-Lingual Transfer of Large Language Models

Analyzing the Evaluation of Cross-Lingual Knowledge Transfer in Multilingual Language Models

Recent advances in training multilingual language models on large datasets seem to have shown promis

BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfer

Despite remarkable advancements in few-shot generalization in natural language processing, most mode

And Where that Language Does Not Yet Exist

Abstract This essay-review is centered on two recently published contemporary works

Effects of Language Relatedness for Cross-lingual Transfer Learning in Character-Based Language Models

Character-based Neural Network Language Models (NNLM) have the advantage of smaller vocabulary and t

When Does Language Transfer Help? Sequential Fine-Tuning for Cross-Lingual Euphemism Detection

Euphemisms are culturally variable and often ambiguous, posing challenges for language models, espec

Code-Switching In-Context Learning for Cross-Lingual Transfer of Large Language Models

While large language models (LLMs) exhibit strong multilingual abilities, their reliance on English