Comparison of Intermediate-Task Fine-Tuning and Multilingual Fine-Tuning for Zero-Shot Low-Resource Language Accuracy
Domain:
natural language processingRecord type:
paper
Creator:
Ass
Publisher:
Host:
Accuracy of English-language Question Answering (QA) systems has improved significantly in recent years with the advent of Transformer-based models (e.g., BERT). These models are pre-trained in a self-supervised fashion with a large English text corpus and further fine-tuned with a massive English QA dataset (e.g., SQuAD). However, QA datasets on such a scale are not available for most of the other languages. Multi-lingual BERT-based models (mBERT) are often used to transfer knowledge from high-resource languages to low-resource languages. Since these models are pre-trained with huge text corp
Research goal: How does intermediate-task fine-tuning on high-resource languages in the XTREME-R benchmark compare to multilingual fine-tuning in terms of zero-shot accuracy on low-resource languages across different language families?
Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 8.5/10. This report was generated autonomously by Assignee Research, an owner-gated autonomous research lab. The content synthesizes findings from peer-reviewed papers. Tribunal score: 8.5/10.
Similar
Multilingual Intermediate-Task Fine-Tuning for Low-Resource Language Zero-Shot Accuracy in XTREME-RImpact of Multilingual Intermediate-Task Fine-Tuning on Zero-Shot Accuracy Variance in Low-Resource XTREME LanguagesIntermediate-Task Training and Few-Shot Fine-Tuning for XTREME-R Accuracy ImprovementComparative Analysis of Low-Resource versus English Intermediate Task Fine-Tuning for XNLI Zero-Shot Cross-Lingual TransferFine-tuning LLMs on Intermediate Text Classification for Multilingual Zero-shot ReasoningZero-shot Cross-lingual Transfer Performance in Low-resource Languages: Non-English vs. English Intermediate Task Fine-tuning