Logo Lanfrica
  • Accueil
  • Atlas
  • Analyses
  • Documentation
  • Sign in

© 2026 Lanfrica. Tous droits réservés. Tous les droits d'auteur des ressources affichées sur le site Web Lanfrica appartiennent aux détenteurs de droits d'auteur d'origine, sauf indication contraire explicite.

MaLA-500: Massive Language Adaptation of Large Language Models

Domaine:

natural language processing

Type de record:

papermodel
Créateur:
LinJi,TieMar
Hôte:avatar
Large language models (LLMs) have advanced the state of the art in natural language processing. However, their predominant design for English or a limited set of languages creates a substantial gap in their effectiveness for low-resource languages. To bridge this gap, we introduce MaLA-500, a novel large language model designed to cover an extensive range of 534 languages. To train MaLA-500, we employ vocabulary extension and continued pretraining on LLaMA 2 with Glot500-c. Our intrinsic evaluation demonstrates that MaLA-500 is better at predicting the given texts of low-resource languages than existing multilingual LLMs. Moreover, the extrinsic evaluation of in-context learning shows that MaLA-500 outperforms previous LLMs on SIB200 and Taxi1500 by a significant margin, i.e., 11.68% and 4.82% marco-average accuracy across languages. We release MaLA-500 at huggingface.co

Visit

arxiv.org

Tasks

language modeling

Languages

Mala

Tags

Computation and Language

Similaires

EMMA-500: Enhancing Massively Multilingual Adaptation of Large Language ModelsRomanization-based Large-scale Adaptation of Multilingual Language ModelsMassively Multilingual Adaptation of Large Language Models Using Bilingual Translation DataLow-Resource Dialect Adaptation of Large Language Models: A French Dialect Case-StudyLangCompress: Language-Aware Compression of Large Language ModelsGlot500: Scaling Multilingual Corpora and Language Models to 500 Languages

EMMA-500: Enhancing Massively Multilingual Adaptation of Large Language Models

In this work, we introduce EMMA-500, a large-scale multilingual language model continue-trained on t

Romanization-based Large-scale Adaptation of Multilingual Language Models

Large multilingual pretrained language models (mPLMs) have become the de facto state of the art for

Massively Multilingual Adaptation of Large Language Models Using Bilingual Translation Data

This paper investigates a critical design decision in the practice of massively multilingual continu

Low-Resource Dialect Adaptation of Large Language Models: A French Dialect Case-Study

Despite the widespread adoption of Large Language Models (LLMs), their strongest capabilities remain

LangCompress: Language-Aware Compression of Large Language Models

Large Language Models (LLMs) demonstrate strong multilingual capabilities but are costly to deploy d

Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages

The NLP community has mainly focused on scaling Large Language Models (LLMs) vertically, i.e., makin