Logo Lanfrica
  • Accueil
  • Atlas
  • Analyses
  • Documentation
  • Sign in

© 2026 Lanfrica. Tous droits réservés. Tous les droits d'auteur des ressources affichées sur le site Web Lanfrica appartiennent aux détenteurs de droits d'auteur d'origine, sauf indication contraire explicite.

WEAM Pre-training for Robust Zero-Shot Cross-Lingual Transfer in Niger-Congo Languages

Domaine:

natural language processing

Type de record:

paper
Créateur:
Ass
Éditeur:
Zenodo
Hôte:avatar
Multilingual BERT (mBERT), a language model pre-trained on large multilingual corpora, has impressive zero-shot cross-lingual transfer capabilities and performs surprisingly well on zero-shot POS tagging and Named Entity Recognition (NER), as well as on cross-lingual model transfer. At present, the mainstream methods to solve the cross-lingual downstream tasks are always using the last transformer layer's output of mBERT as the representation of linguistic information. In this work, we explore the complementary property of lower layers to the last transformer layer of mBERT. A feature aggregat Research goal: Does the WEAM pre-training strategy improve robustness against typological divergence in zero-shot cross-lingual transfer performance on XNLI for Niger-Congo languages relative to standard multilingual BERT baselines? Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 8.5/10. This report was generated autonomously by Assignee Research, an owner-gated autonomous research lab. The content synthesizes findings from peer-reviewed papers. Tribunal score: 8.5/10.

Visit

doi.orgzenodo.org

Tags

WEAMpre-trainingstrategyimproverobustnessagainsttypologicaldivergence

Licenses

Creative Commons Attribution 4.0 Internationalhttps://creativecommons.org/licenses/by/4.0/legalcode

Similaires

Multilingual Adversarial Training for Robust Zero-Shot Cross-Lingual Transfer in Low-Resource LanguagesMultilingual Intermediate-Task Training for Robust Zero-Shot Cross-Lingual Transfer in Low-Resource LanguagesContrastive Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer in Low-Resource LanguagesScaling Intermediate Languages for Robust Zero-Shot Cross-Lingual Transfer in XTREMEJoint Intermediate-Task Training for Robust Zero-Shot Cross-Lingual Transfer on XTREMEIntermediate-Task Training for Robust Zero-Shot Cross-Lingual Transfer to Low-Resource Languages in XTREME-R

Multilingual Adversarial Training for Robust Zero-Shot Cross-Lingual Transfer in Low-Resource Languages

Pre-trained multilingual language encoders, such as multilingual BERT and XLM-R, show great potentia

Multilingual Intermediate-Task Training for Robust Zero-Shot Cross-Lingual Transfer in Low-Resource Languages

Pre-trained multilingual language encoders, such as multilingual BERT and XLM-R, show great potentia

Contrastive Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer in Low-Resource Languages

This paper studies zero-shot cross-lingual transfer of vision-language models. Specifically, we focu

Scaling Intermediate Languages for Robust Zero-Shot Cross-Lingual Transfer in XTREME

Intermediate-task training---fine-tuning a pretrained model on an intermediate task before fine-tuni

Joint Intermediate-Task Training for Robust Zero-Shot Cross-Lingual Transfer on XTREME

Pre-trained multilingual language encoders, such as multilingual BERT and XLM-R, show great potentia

Intermediate-Task Training for Robust Zero-Shot Cross-Lingual Transfer to Low-Resource Languages in XTREME-R

Intermediate-task training---fine-tuning a pretrained model on an intermediate task before fine-tuni