Logo Lanfrica
  • Home
  • Atlas
  • Insights
  • Docs
  • Sign in

© 2026 Lanfrica. All rights reserved. All copyrights of the resources shown on the Lanfrica website belong to the original copyright holders, unless explicitly stated otherwise.

Gradient Clipping Thresholds in XLM-R Fine-Tuning for Zero-Shot Cross-Lingual Transfer

Domain:

natural language processing

Record type:

paper
Creator:
Ass
Publisher:
Zenodo
Host:avatar
Pre-trained multilingual language encoders, such as multilingual BERT and XLM-R, show great potential for zero-shot cross-lingual transfer. However, these multilingual encoders do not precisely align words and phrases across languages. Especially, learning alignments in the multilingual embedding space usually requires sentence-level or word-level parallel corpora, which are expensive to be obtained for low-resource languages. An alternative is to make the multilingual encoders more robust; when fine-tuning the encoder using downstream task, we train the encoder to tolerate noise in the contex Research goal: What is the impact of gradient clipping thresholds during fine-tuning on the zero-shot cross-lingual transfer performance of XLM-R Base, and how do different thresholds compare in terms of downstream task accuracy on XTREME-R? Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 7.5/10. This report was generated autonomously by Assignee Research, an owner-gated autonomous research lab. The content synthesizes findings from peer-reviewed papers. Tribunal score: 7.5/10.

Visit

doi.org

Tasks

transfer learning

Tags

impactgradientclippingthresholdsduringfine-tuningzero-shotcross-lingual

Licenses

Creative Commons Attribution 4.0 Internationalhttps://creativecommons.org/licenses/by/4.0/legalcode

Similar

Gradient-based adversarial fine-tuning for XLM-R robustness in zero-shot cross-lingual classificationContrastive-Adversarial Fine-Tuning for Zero-Shot Cross-Lingual Performance in XLM-RContrastive Learning Effects on Zero-Shot Cross-Lingual Euphemism Detection in XLM-R Fine-TuningSequential Fine-Tuning Task Quantity and Zero-Shot Cross-Lingual Transfer in XTREME-RHybrid Batch Training for XLM-R in Zero-Shot Cross-Lingual TransferMultimodal Alignment Tasks and Zero-Shot Cross-Lingual Transfer in XLM-R

Gradient-based adversarial fine-tuning for XLM-R robustness in zero-shot cross-lingual classification

Pre-trained multilingual language encoders, such as multilingual BERT and XLM-R, show great potentia

Contrastive-Adversarial Fine-Tuning for Zero-Shot Cross-Lingual Performance in XLM-R

Pre-trained multilingual language encoders, such as multilingual BERT and XLM-R, show great potentia

Contrastive Learning Effects on Zero-Shot Cross-Lingual Euphemism Detection in XLM-R Fine-Tuning

Euphemisms are culturally variable and often ambiguous, posing challenges for language models, espec

Sequential Fine-Tuning Task Quantity and Zero-Shot Cross-Lingual Transfer in XTREME-R

Euphemisms are culturally variable and often ambiguous, posing challenges for language models, espec

Hybrid Batch Training for XLM-R in Zero-Shot Cross-Lingual Transfer

Pre-trained multilingual language encoders, such as multilingual BERT and XLM-R, show great potentia

Multimodal Alignment Tasks and Zero-Shot Cross-Lingual Transfer in XLM-R

The introduction of pretrained cross-lingual language models brought decisive improvements to multil