aliakarma/trail-nlp

TRAIL: Trust-Aware Cross-Lingual Alignment for Reliable LLMs in Low-Resource Languages # TRAIL: Trust-Aware Cross-Lingual Alignment for Reliable LLMs in Low-Resource Languages --- ## TL;DR Large language models are both miscalibrated and brittle for low-resource languages like Saraiki, Sindhi, and Pashto. TRAIL jointly optimises cross-lingual alignment, probabilistic calibration, and entropy regularisation in a single fine-tuning objective, reducing Expected Calibration Error by over 66% (0.18 → 0.06 for XLM-R) while improving cross-lingual consistency by 16 percentage points. To use a TRAIL-trained model with trust-gated inference in three lines: ```python from trail import TrustGatedPredictor predictor = TrustGatedPredictor.from_pretrained("trail-nlp/trail-xlmr-saraiki") predictions, confidences, abstain_mask = predictor.predict(input_ids, attention_mask) ``` --- ## Abstract Large language models (LLMs) have achieved impressive performance on high-resource NLP benchmarks, yet their deployment in low-resource language settings remains limited by two compounding deficiencies: poor cross-lingual transfer fidelity and systematic miscalibration. Miscalibrated models produce confidence scores that are systematically inconsistent with empirical accuracy — a critical failure mode in high-stakes applications where expressed uncertainty must be trusted. These problems intensify in low-resource languages such as Saraiki, Sindhi, and Pashto, where data scarcity constrains both the learning signal and the diagnostic evaluation. We introduce **TRAIL** (Trust-Aware Representation and Alignment for Low-resource LLMs), a principled multi-objective training framework that jointly addresses cross-lingual representation alignment and probabilistic calibration. TRAIL augments the standard task loss with three theoretically motivated auxiliary terms: (i) a *cosine alignment loss*; (ii) a *differentiable calibration surrogate* (MMCE); and (iii) an *entropy regulariser*. We further introduce a *Trust Index* (TI) and a *trust-gated inference* rule. Experiment …