Beyond Generalization :Evaluating Multilingual LLMs for Yorùbá Animal Health Translation
Domaine:
natural language processingType de record:
datasetpaper
Créateur:
AssAdeAdeAde
Éditeur:
Und
Hôte:
Machine translation (MT) has advanced significantly for high-resource languages, yet specialized domain translation remains a challenge for low-resource languages. This study evaluates the abilityof state-of-the-art multilingual models to translate animal health reports from English to Yorùbá, a crucial task for veterinary communication in underserved regions. We curated a dataset of 1,468 parallel sentences and compared multiple MT models in zero-shot and fine-tuned settings. Our findings indicate substantial limitations in their ability to generalize to domain-specific translation, with common errors arising from vocabulary mismatch, training data scarcity, and morphological complexity. Fine-tuning improves performance, particularly for the NLLB 3.3B model, but challenges remain in preserving technical accuracy. These results underscore the need for more targeted approaches to multilingual and culturally aware LLMs for African languages.
Similaires
Beyond English: Evaluating LLMs for Arabic Grammatical Error CorrectionEvaluating Robustness of LLMs to Typographical Noise in Yorùbá QACricBench: A Multilingual Benchmark for Evaluating LLMs in Cricket AnalyticsXTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual GeneralizationLLMs Beyond English: Scaling the Multilingual Capability of LLMs with Cross-Lingual FeedbackNsanku: Evaluating Zero-Shot Translation Performance of LLMs for Ghanaian Languages