Logo Lanfrica
  • Accueil
  • Atlas
  • Analyses
  • Documentation
  • Sign in

© 2026 Lanfrica. Tous droits réservés. Tous les droits d'auteur des ressources affichées sur le site Web Lanfrica appartiennent aux détenteurs de droits d'auteur d'origine, sauf indication contraire explicite.

SEATauBench: Adapting Tool-Agent-User Evaluation Into Low-Resource Southeast Asian Languages

Domaine:

natural language processing

Type de record:

datasetpaper
Créateur:
NguAdiRuaLee
Éditeur:
arXiv
Hôte:avatar
While AI development and evaluation for Southeast Asia (SEA) has grown rapidly, agent capabilities in regional languages are still poorly understood despite its importance to sovereign AI. To fill this gap, we introduce SEATauBench, the first agent-focused evaluation framework for SEA sovereign AI. SeaTau adapts TauBench to five languages -- Mandarin, Vietnamese, Thai, Indonesian, and Filipino -- and evaluates agents across progressively localized settings that vary the language of user-agent interaction, tool specifications, and task domains. Across three recent models, we find that English agent capabilities transfer reasonably well when only the conversation language changes, but quality and robustness degrade sharply as more task contexts are localized, with the largest losses in full domain adaptation. We also the limits of English-only agent assessment for measuring agent capabilities in SEA languages. More broadly, SeaTau provides a diagnostic benchmark and reusable adaptation pipeline for building reliable multilingual agents for linguistically diverse regions. Data and code can be accessed at github.com. 23 pages

Visit

doi.orgarxiv.org

Tags

Computation and Language (cs.CL)Artificial Intelligence (cs.AI)FOS: Computer and information sciences

Licenses

Creative Commons Attribution 4.0 Internationalhttps://creativecommons.org/licenses/by/4.0/legalcode

Similaires

User-friendly automatic transcription of low-resource languages: Plugging ESPnet into ElpisAdapting to the Low-Resource Double-Bind: Investigating Low-Compute Methods on Low-Resource African LanguagesApplause: A Learning Tool for Low-Resource Languagesnaolbakala/Adapting-Outperformer-Topic-Modeling-for-low-resource-Major-Ethiopian-LanguagesAdapting Multilingual LLMs to Low-Resource Languages with Knowledge Graphs via AdaptersSAGE: Sustainable Agent-Guided Expert-tuning for Culturally Attuned Translation in Low-Resource Southeast Asia

User-friendly automatic transcription of low-resource languages: Plugging ESPnet into Elpis

International audience This paper reports on progress integrating the speech recognit

Adapting to the Low-Resource Double-Bind: Investigating Low-Compute Methods on Low-Resource African Languages

Many natural language processing (NLP) tasks make use of massively pre-trained language models, whic

Applause: A Learning Tool for Low-Resource Languages

In this paper, we describe the concept of an interactive tool which can be employed to build a dialo

naolbakala/Adapting-Outperformer-Topic-Modeling-for-low-resource-Major-Ethiopian-Languages

Topic words based dataset extracted from large size of social media data from three major Ethiopian

Adapting Multilingual LLMs to Low-Resource Languages with Knowledge Graphs via Adapters

This paper explores the integration of graph knowledge from linguistic ontologies into multilingual

SAGE: Sustainable Agent-Guided Expert-tuning for Culturally Attuned Translation in Low-Resource Southeast Asia

The vision of an inclusive World Wide Web is impeded by a severe linguistic divide, particularly for