surrey-nlp/ALOPE

[COLM2025] Repository contains codebase for "ALOPE" framework which can help train efficient LoRA adapters for Quality Estimation of Machine Translated Text. Additionally, it also support our [LowResLM @ EACL 2026] paper on Domain-specific QE for MT in low-resource scenarios. # ALOPE: Adaptive Layer Optimization for Translation Quality Estimation **Published at**: COLM 2025 • arXiv ## Overview **ALOPE** is an adaptive layer-optimization framework that enhances Quality Estimation (QE) for machine translation using large language models. QE evaluates the quality of a source–target translation pair **without reference translations**, making it essential for real-world MT deployment. ALOPE restructures Transformer representations through **layer-wise adaptation**, integrating **LoRA (Low-Rank Adaptation)** with regression task heads on selected intermediate Transformer layers. The key insight is that **intermediate Transformer layers provide superior cross-lingual representations** compared to the commonly used final layer. ### Key Findings - **Intermediate layers outperform the final layer**: Transformer Layer −7 (TL-7) delivers the best overall QE performance across models and language pairs, with TL-11 as a strong alternative. - **Three complementary strategies**: Single-layer regression, dynamic weighting, and multi-head regression offer flexibility depending on deployment constraints. - **Consistent improvements over baselines**: ALOPE achieves results comparable to established QE frameworks while being parameter-efficient through 4-bit QLoRA quantization. - **Strong cross-lingual transfer**: Effective across 8 language pairs spanning both English→Indic and Indic→English directions. --- ## Three ALOPE Strategies ### 1. Single-Layer Regression Extracts hidden representations from a **single intermediate Transformer layer** and trains a regression head on top for DA (Direct Assessment) score prediction. This is the simplest and most interpretable approach — used to identify which layers carry the most QE-relevant information. ### 2. Dynamic Weighting Adaptively **combines representations from multiple Transformer layers** using learned weights. Rather than committing to a single layer, this strategy lets the model di …