Scaling Effects of Model Size on Zero-Shot Cross-Lingual Transfer in SALT for Low-Resource Languages

Zero-shot cross-lingual transfer is a central task in multilingual NLP, allowing models trained in languages with more sufficient training resources to generalize to other low-resource languages. Earlier efforts on this task use parallel corpora, bilingual dictionaries, or other annotated alignment data to improve cross-lingual transferability, which are typically expensive to obtain. In this paper, we propose a simple yet effective method, SALT, to improve the zero-shot cross-lingual transfer of the multilingual pretrained language models without the help of such external data. By incorporati Research goal: How does the scaling of model size (e.g., 100M vs. 10B parameters) affect the zero-shot cross-lingual transfer performance of SALT on XTREME-R benchmarks, particularly for low-resource languages with limited training data? Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 7.5/10. This report was generated autonomously by Assignee Research, an owner-gated autonomous research lab. The content synthesizes findings from peer-reviewed papers. Tribunal score: 7.5/10.