Logo Lanfrica
  • Accueil
  • Atlas
  • Analyses
  • Documentation
  • Sign in

© 2026 Lanfrica. Tous droits réservés. Tous les droits d'auteur des ressources affichées sur le site Web Lanfrica appartiennent aux détenteurs de droits d'auteur d'origine, sauf indication contraire explicite.

Multilingual language model Adaptive Fine-Tuning: A Study on African Languages

Domaine:

natural language processing

Type de record:

paper
Multilingual pre-trained language models (PLMs) have demonstrated impressive performance on several downstream tasks on both high-resourced and low-resourced languages. However, there is still a large performance drop for languages unseen during pre-training, especially African languages. One of the most effective approaches to adapt to a new language is language adaptive fine-tuning (LAFT) — fine-tuning a multilingual PLM on monolingual texts using the pre-training objective. However, African languages with large monolingual texts are few, and adapting to each of them individually takes large disk space and limits the cross-lingual transfer abilities of the resulting models because they have been specialized for a single language. In this paper, we perform multilingual adaptive fine-tuning (MAFT) on 17 most-resourced African languages and three other high-resource languages widely spoken on the continent – English, French, and Arabic to encourage cross-lingual transfer learning. Additionally, to further specialize the multilingual PLM, we removed vocabulary tokens from the embedding layer that corresponds to non-African writing scripts before MAFT, thus reducing the model size by 50%. Our evaluation on two multilingual PLMs (AfriBERTa and XLM-R) and three NLP tasks (NER, news topic classification, and sentiment classification) shows that our approach is competitive to applying LAFT on individual languages while requiring significantly less disk space.

Visit

arxiv.orgopenreview.net

Connected records

datasetmodel

Tasks

topic classificationtransfer learning

Languages

AfrikaansAmharicChichewaHausaIgboKinyarwandaMalagasyOromoPidgin, NigerianRundi+7

Tags

africanlp2

Similaires

Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-TuningMULTILINGUAL ADAPTIVE FINE-TUNING (MAFT)Adapting Multilingual Speech Representation Model for a New, Underresourced Language through Multilingual Fine-tuning and Continued PretrainingIncreasing linguistic diversity in NLP : Fine-tuning Multilingual Pre-trained African Language ModelsComparison of Intermediate-Task Fine-Tuning and Multilingual Fine-Tuning for Zero-Shot Low-Resource Language AccuracyInvestigating the Impact of Language-Adaptive Fine-Tuning on Sentiment Analysis in Hausa Language Using AfriBERTa

Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuning

Multilingual pre-trained language models (PLMs) have demonstrated impressive performance on several downstream tasks for both high-resourced and low-resourced languages. However, there is still a large performance drop for languages unseen during pre-training, espe

MULTILINGUAL ADAPTIVE FINE-TUNING (MAFT)

We introduce MAFT as an approach to adapt a multi-lingual PLM to a new set of languages. Adapting PLMs has been shown to be effective when adapting to a new domain (Gururangan et al., 2020) or language (Pfeiffer et al., 2020; Alabi et al., 2020; Adelani et al., 202

Adapting Multilingual Speech Representation Model for a New, Underresourced Language through Multilingual Fine-tuning and Continued Pretraining

In recent years, neural models learned through self-supervised pretraining on large scale multilingu

Increasing linguistic diversity in NLP : Fine-tuning Multilingual Pre-trained African Language Models

Increasing linguistic diversity in NLP : Fine-tuning Multilingual Pre-trained African Language Models

Poster presented at the Deep Learning Indaba 2023 by Fiskani Banda

Comparison of Intermediate-Task Fine-Tuning and Multilingual Fine-Tuning for Zero-Shot Low-Resource Language Accuracy

Accuracy of English-language Question Answering (QA) systems has improved significantly in recent ye

Investigating the Impact of Language-Adaptive Fine-Tuning on Sentiment Analysis in Hausa Language Using AfriBERTa

Sentiment analysis (SA) plays a vital role in Natural Language Processing (NLP) by ~identifying sent