Logo Lanfrica
  • Home
  • Atlas
  • Insights
  • Docs
  • Sign in

© 2026 Lanfrica. All rights reserved. All copyrights of the resources shown on the Lanfrica website belong to the original copyright holders, unless explicitly stated otherwise.

Meta-Pretraining for Zero-Shot Cross-Lingual Named Entity Recognition in Low-Resource Philippine Languages

Domain:

natural language processing

Record type:

papermodel
Creator:
AfrSalWeiBut
Host:avatar
Named-entity recognition (NER) in low-resource languages is usually tackled by finetuning very large multilingual LMs, an option that is often infeasible in memory- or latency-constrained settings. We ask whether small decoder LMs can be pretrained so that they adapt quickly and transfer zero-shot to languages unseen during pretraining. To this end we replace part of the autoregressive objective with first-order model-agnostic meta-learning (MAML). Tagalog and Cebuano are typologically similar yet structurally different in their actor/non-actor voice systems, and hence serve as a challenging test-bed. Across four model sizes (11 M - 570 M) MAML lifts zero-shot micro-F1 by 2-6 pp under head-only tuning and 1-3 pp after full tuning, while cutting convergence time by up to 8%. Gains are largest for single-token person entities that co-occur with Tagalog case particles si/ni, highlighting the importance of surface anchors. Accepted (poster) to 5th Workshop on Multilingual Representation Learning at EMNLP 2025

Visit

arxiv.org

Tasks

information extractionnamed entity recognitiontransfer learning

Tags

Computation and LanguageArtificial Intelligence

Similar

Named Entity Recognition in Low-resource Languages using Cross-lingual distributional word representationCROP: Zero-shot Cross-lingual Named Entity Recognition with Multilingual Labeled Sequence TranslationMultilingual Pretraining and Intermediate-Task Training for Zero-Shot Cross-Lingual Transfer in Low-Resource LanguagesPerformance comparison of cross-lingual transfer learning methods for Named Entity Recognition in low-resource African languagesMultimodal Pretraining as Intermediate Tasks for Zero-Shot Cross-Lingual Transfer in Low-Resource Languages on XTREME-RQuantized LLMs for Zero-Shot Cross-Lingual Reasoning in Low-Resource Languages

Named Entity Recognition in Low-resource Languages using Cross-lingual distributional word representation

Named Entity Recognition (NER) is a fundamental task in many NLP applications that seek to identify

CROP: Zero-shot Cross-lingual Named Entity Recognition with Multilingual Labeled Sequence Translation

Named entity recognition (NER) suffers from the scarcity of annotated training data, especially for

Multilingual Pretraining and Intermediate-Task Training for Zero-Shot Cross-Lingual Transfer in Low-Resource Languages

Intermediate-task training---fine-tuning a pretrained model on an intermediate task before fine-tuni

Performance comparison of cross-lingual transfer learning methods for Named Entity Recognition in low-resource African languages

Cross-lingual transfer learning enables NLP for low-resource languages by leveraging labeled data fr

Multimodal Pretraining as Intermediate Tasks for Zero-Shot Cross-Lingual Transfer in Low-Resource Languages on XTREME-R

Intermediate-task training---fine-tuning a pretrained model on an intermediate task before fine-tuni

Quantized LLMs for Zero-Shot Cross-Lingual Reasoning in Low-Resource Languages

Intermediate-task training---fine-tuning a pretrained model on an intermediate task before fine-tuni