MERLIN: Multi-Stage Curriculum Alignment for Multilingual Encoder-LLM Integration in Cross-Lingual Reasoning

Domaine:

natural language processing

Type de record:

paper

Créateur:

UemGuzNguAla

Hôte:

Large language models excel in English but still struggle with complex reasoning in many low-resource languages (LRLs). Existing encoder-plus-decoder methods such as LangBridge and MindMerger raise accuracy on mid and high-resource languages, yet they leave a large gap on LRLs. We present MERLIN, a two-stage model-stacking framework that applies a curriculum learning strategy -- from general bilingual bitext to task-specific data -- and adapts only a small set of DoRA weights. On the AfriMGSM benchmark MERLIN improves exact-match accuracy by +12.9 pp over MindMerger and outperforms GPT-4o-mini. It also yields consistent gains on MGSM and MSVAMP (+0.9 and +2.8 pp), demonstrating effectiveness across both low and high-resource settings. Accepted to EACL 2026 (main conference)

Visit

arxiv.org

Similaires

Robust Multilingual Encoder Training for Low-Resource Language AlignmentOptimal Transport Distillation for Zero-Shot Cross-Lingual Reasoning in Multilingual LLMsCross-Lingual Alignment via Targeted Lexical Injection in Lugha-Llama for Zero-Shot Swahili ReasoningWord Alignment Integration in Cross-Lingual Sentence Embeddings for XNLI Zero-Shot TransferMassively Multilingual Document Alignment with Cross-lingual Sentence-Mover's DistanceCross-lingual Contrastive Loss Effects on Multilingual Embedding Alignment Quality

Robust Multilingual Encoder Training for Low-Resource Language Alignment

Pre-trained multilingual language encoders, such as multilingual BERT and XLM-R, show great potentia

Optimal Transport Distillation for Zero-Shot Cross-Lingual Reasoning in Multilingual LLMs

Benefiting from transformer-based pre-trained language models, neural ranking models have made signi

Cross-Lingual Alignment via Targeted Lexical Injection in Lugha-Llama for Zero-Shot Swahili Reasoning

Large Language Models (LLMs) have demonstrated remarkable capabilities, yet their performance in low

Word Alignment Integration in Cross-Lingual Sentence Embeddings for XNLI Zero-Shot Transfer

We propose a new approach for learning contextualised cross-lingual word embeddings based on a small

Massively Multilingual Document Alignment with Cross-lingual Sentence-Mover's Distance

Document alignment aims to identify pairs of documents in two distinct languages that are of compara

Cross-lingual Contrastive Loss Effects on Multilingual Embedding Alignment Quality

Existing zero-shot cross-lingual transfer methods rely on parallel corpora or bilingual dictionaries