BAD-X: Bilingual Adapters Improve Zero-Shot Cross-Lingual Transfer

Domain:

natural language processing

Record type:

paper

Adapter modules enable modular and efficient zero-shot cross-lingual transfer, where current state-of-the-art adapter-based approaches learn specialized language adapters (LAs) for individual languages. In this work, we show that it is more effective to learn bilingual language pair adapters (BAs) when the goal is to optimize performance for a particular source-target transfer direction. Our novel BAD-X adapter framework trades off some modularity of dedicated LAs for improved transfer performance: we demonstrate consistent gains in three standard downstream tasks, and for the majority of evaluated low-resource languages.

Visit

aclanthology.org ACL ARR 2022 January Blind Submission

Tasks

dependency parsing natural language inference part of speech tagging transfer learning

Orthogonal Language and Task Adapters in Zero-Shot Cross-Lingual Transfer

Adapter modules, additional trainable parameters that enable efficient fine-tuning of pretrained tra

Adversarial Intermediate Task Training for Robust Zero-Shot Cross-Lingual Transfer on Perturbed PAWS-X Inputs

Pre-trained multilingual language encoders, such as multilingual BERT and XLM-R, show great potentia

Bilingual Lexicons for Zero-Shot Cross-Lingual Retrieval in Low-Resource Languages

Transferring information retrieval (IR) models from a high-resource language (typically English) to

Zero-Shot Transfer Learning using Affix and Correlated Cross-Lingual Embeddings.

Learning morphologically supplemented embedding spaces using cross-lingual models has become an acti

Intermediate-Task Training for Low-Resource Zero-Shot Cross-Lingual Transfer

Intermediate-task training---fine-tuning a pretrained model on an intermediate task before fine-tuni

Adversarial Training Effects on Zero-Shot Cross-Lingual Transfer in XTREME