Logo Lanfrica
  • Accueil
  • Atlas
  • Analyses
  • Documentation
  • Sign in

© 2026 Lanfrica. Tous droits réservés. Tous les droits d'auteur des ressources affichées sur le site Web Lanfrica appartiennent aux détenteurs de droits d'auteur d'origine, sauf indication contraire explicite.

Scaling Model Size and Zero-Shot Transfer to Low-Resource Languages in XTREME

Domaine:

natural language processing

Type de record:

paper
Créateur:
Ass
Éditeur:
Zenodo
Hôte:avatar
Intermediate-task training---fine-tuning a pretrained model on an intermediate task before fine-tuning again on the target task---often improves model performance substantially on language understanding tasks in monolingual English settings. We investigate whether English intermediate-task training is still helpful on non-English target tasks. Using nine intermediate language-understanding tasks, we evaluate intermediate-task transfer in a zero-shot cross-lingual setting on the XTREME benchmark. We see large improvements from intermediate training on the BUCC and Tatoeba sentence retrieval tas Research goal: What is the impact of scaling model size from 1B to 175B parameters on the efficacy of English-only intermediate-task training for zero-shot transfer to low-resource languages in the XTREME suite? Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 9.2/10. This report was generated autonomously by Assignee Research, an owner-gated autonomous research lab. The content synthesizes findings from peer-reviewed papers. Tribunal score: 9.2/10.

Visit

doi.org

Tasks

transfer learning

Tags

impactscalingmodelsizeparametersefficacyEnglish-onlyintermediate-task

Licenses

Creative Commons Attribution 4.0 Internationalhttps://creativecommons.org/licenses/by/4.0/legalcode

Similaires

Scaling Model Size and Zero-Shot Cross-Lingual Transfer Performance in Low-Resource LanguagesScaling Model Size for Zero-Shot Cross-Lingual Transfer in Low-Resource Settings on XTREME-RScaling Pretrained Model Size and Intermediate-Task Training for Zero-Shot Transfer in Low-Resource LanguagesScaling Model Size and Intermediate-Task Training for Zero-Shot Cross-Lingual Transfer in Low-Resource LanguagesScaling Effects of Model Size on Zero-Shot Cross-Lingual Transfer in SALT for Low-Resource LanguagesScaling Model Size Effects on Zero-Shot Cross-Lingual Transfer in XTREME Benchmark

Scaling Model Size and Zero-Shot Cross-Lingual Transfer Performance in Low-Resource Languages

Pre-trained multilingual language encoders, such as multilingual BERT and XLM-R, show great potentia

Scaling Model Size for Zero-Shot Cross-Lingual Transfer in Low-Resource Settings on XTREME-R

Intermediate-task training---fine-tuning a pretrained model on an intermediate task before fine-tuni

Scaling Pretrained Model Size and Intermediate-Task Training for Zero-Shot Transfer in Low-Resource Languages

Intermediate-task training---fine-tuning a pretrained model on an intermediate task before fine-tuni

Scaling Model Size and Intermediate-Task Training for Zero-Shot Cross-Lingual Transfer in Low-Resource Languages

Intermediate-task training---fine-tuning a pretrained model on an intermediate task before fine-tuni

Scaling Effects of Model Size on Zero-Shot Cross-Lingual Transfer in SALT for Low-Resource Languages

Zero-shot cross-lingual transfer is a central task in multilingual NLP, allowing models trained in l

Scaling Model Size Effects on Zero-Shot Cross-Lingual Transfer in XTREME Benchmark

Intermediate-task training---fine-tuning a pretrained model on an intermediate task before fine-tuni