Logo Lanfrica
  • Accueil
  • Atlas
  • Analyses
  • Documentation
  • Sign in

© 2026 Lanfrica. Tous droits réservés. Tous les droits d'auteur des ressources affichées sur le site Web Lanfrica appartiennent aux détenteurs de droits d'auteur d'origine, sauf indication contraire explicite.

The Multilingual Evaluation Paradox: Ramoju Potential Across Languages, Scripts, and 2.37 Billion Speakers

Domaine:

natural language processing

Type de record:

paper
Créateur:
RamAnuMudRam
Éditeur:
Zenodo
Hôte:avatar
Technical Report XARL-2026-06. We evaluate Llama-3-8B-Instruct on the Multilingual GSM8K (MGSM) benchmark across seven languages covering 2.37 billion native speakers and find that every non-English language exhibits higher Ramoju Potential than English without exception. We introduce the Multilingual Suppression Index (MSI), finding Chinese reaches MSI=2.87x — meaning 1.4 billion Chinese speakers face an evaluation gap nearly three times larger than English speakers. We identify three instruction-response patterns: instruction-hurt (English), instruction-positive (Chinese, Japanese, Bengali, Swahili, Telugu), and instruction-amplifying (Russian). We establish the Multilingual Evaluation Paradox: format suppression scales inversely with language resource richness, compounding linguistic inequality in AI evaluation. Sixth in the XARL evaluation series.

Visit

doi.orgzenodo.org

Languages

Swahili

Tags

multilingual evaluationRamoju PotentialMultilingual Suppression Indexlanguage modelsMGSMAI fairnessbenchmark methodologyformat compliancemultilingual NLPevaluation paradox

Licenses

Creative Commons Attribution 4.0 Internationalhttps://creativecommons.org/licenses/by/4.0/legalcodeCopyright (C) 2026 The Authors. xArch AI Research Laboratory (XARL). All rights reserved.http://rightsstatements.org/vocab/InC/1.0/

Similaires

Massively Multilingual ASR: 50 Languages, 1 Model, 1 Billion ParametersAWED-FiNER: Agents, Web applications, and Expert Detectors for Fine-grained Named Entity Recognition across 36 Languages for 6.6 Billion SpeakersSawtone: A universal framework for phonetic similarity and alignment across languages and scriptsDocAtlas: Multilingual Document Understanding Across 80+ LanguagesDETECTING CYBERBULLYING ACROSS NIGERIAN LANGUAGES: A MULTILINGUAL SYSTEMTowards Scalable Multilingual AI:Benchmarking CompressedTransformer Models Across Languages

Massively Multilingual ASR: 50 Languages, 1 Model, 1 Billion Parameters

We study training a single acoustic model for multiple languages with the aim of improving automatic

AWED-FiNER: Agents, Web applications, and Expert Detectors for Fine-grained Named Entity Recognition across 36 Languages for 6.6 Billion Speakers

Named Entity Recognition (NER) is a foundational task in Natural Language Processing (NLP) and Infor

Sawtone: A universal framework for phonetic similarity and alignment across languages and scripts

Processing text across different scripts presents significant hurdles in natural language processing

DocAtlas: Multilingual Document Understanding Across 80+ Languages

Multilingual document understanding remains limited for low-resource languages due to scarce trainin

DETECTING CYBERBULLYING ACROSS NIGERIAN LANGUAGES: A MULTILINGUAL SYSTEM

ABSTRACT

As the digital world evolves, the

Towards Scalable Multilingual AI:Benchmarking CompressedTransformer Models Across Languages

Large Language Models (LLMs) have demonstrated remarkable proficiency in natural language processing