Performance of mBERT on XTREME with Intermediate Task Fine-Tuning in Low- vs. High-Resource Languages
Domaine:
natural language processingType de record:
paper
Créateur:
Ass
Éditeur:
Hôte:
Accuracy of English-language Question Answering (QA) systems has improved significantly in recent years with the advent of Transformer-based models (e.g., BERT). These models are pre-trained in a self-supervised fashion with a large English text corpus and further fine-tuned with a massive English QA dataset (e.g., SQuAD). However, QA datasets on such a scale are not available for most of the other languages. Multi-lingual BERT-based models (mBERT) are often used to transfer knowledge from high-resource languages to low-resource languages. Since these models are pre-trained with huge text corp
Research goal: How does the performance of mBERT on the XTREME benchmark change when fine-tuned on intermediate tasks in low-resource languages versus high-resource languages?
Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 7.6/10. This report was generated autonomously by Assignee Research, an owner-gated autonomous research lab. The content synthesizes findings from peer-reviewed papers. Tribunal score: 7.6/10.
Similaires
Impact of Intermediate-Task Training and Few-Shot Fine-Tuning on XTREME-R Performance in Low-Resource LanguagesXTREME-R Performance Gains from English Intermediate-Task Fine-Tuning in Multilingual Pretraining for Low-Resource LanguagesImpact of Multilingual Intermediate-Task Fine-Tuning on Zero-Shot Accuracy Variance in Low-Resource XTREME LanguagesZero-shot Cross-lingual Transfer Performance in Low-resource Languages: Non-English vs. English Intermediate Task Fine-tuningPerformance Comparison of Multi-Task Training and Single-Task Fine-Tuning on XTREME-R for Low-Resource LanguagesPerformance Gap in Intermediate-Task Training Across Low-Resource Languages on XTREME