A Little Pretraining Goes a Long Way: A Case Study on Dependency Parsing Task for Low-resource Morphologically Rich Languages

Domain:

natural language processing

Record type:

papermodel

Creator:

SanKriGupBeh

Host:

Neural dependency parsing has achieved remarkable performance for many domains and languages. The bottleneck of massive labeled data limits the effectiveness of these approaches for low resource languages. In this work, we focus on dependency parsing for morphological rich languages (MRLs) in a low-resource setting. Although morphological information is essential for the dependency parsing task, the morphological disambiguation and lack of powerful analyzers pose challenges to get this information for MRLs. To address these challenges, we propose simple auxiliary tasks for pretraining. We perform experiments on 10 MRLs in low-resource settings to measure the efficacy of our proposed pretraining method and observe an average absolute gain of 2 points (UAS) and 3.6 points (LAS). Code and data available at: github.com 6 pages, The work is accepted at EACL-SRW, 2021, Kyiv, Ukraine Typos corrected in Section 3.2

Visit

arxiv.org

Tasks

dependency parsing parsing

Intermediate-Task Training on English Syntax for Zero-Shot Parsing in Morphologically Rich Low-Resource Languages

Intermediate-task training---fine-tuning a pretrained model on an intermediate task before fine-tuni

CSSL: Contrastive Self-Supervised Learning for Dependency Parsing on Relatively Free Word Ordered and Morphologically Rich Low Resource Languages

Neural dependency parsing has achieved remarkable performance for low resource morphologically rich

Multilingual Dependency Parsing for Low-Resource {A}frican Languages: Case Studies on {B}ambara, {W}olof, and {Y}oruba

This paper describes a methodology for syntactic knowledge transfer between high-resource languages to extremely low-resource languages. The methodology consists in leveraging multilingual BERT self-attention model pretrained on large datasets to develop a multilin

Multilingual Dependency Parsing for Low-Resource African Languages: Case Studies on Bambara, Wolof, and Yoruba

Pretraining Corpus Domain and Hybrid Multilingual Model Performance in Low-Resource Universal Dependency Parsing

Pretrained multilingual language models have become a common tool in transferring NLP capabilities t

Dependency Parsing Across the Resource Spectrum: Evaluating Architectures on High and Low-Resource Languages

Transformer-based models achieve state-of-the-art dependency parsing for high-resource languages, ye