SPIRAL: Sequential Preference-based Iterative Reinforcement Alignment for Faithful Multi-Document Summarization in Low-Resource Settings

Hallucination poses a critical challenge for multi-document summarization, particularly in low-resource and multilingual settings where limited supervision exacerbates factual inconsistencies. We propose SPIRAL (Sequential Preference-based Iterative Reinforcement Alignment), a lightweight framework that enhances both factual faithfulness and parameter efficiency through progressive reinforcement learning. Our method trains compact models through three sequential stages: supervised cross-entropy pre-training, direct preference optimization (DPO) on automatically constructed preference pairs, and policy-gradient (PG) refinement guided by a multi-component reward signal. Unlike traditional approaches that rely on explicit reward models or computationally intensive reinforcement learning pipelines, SPIRAL directly optimizes preference pairs without a separate reward model, achieving 90% of BART-base quality (ROUGE-1: 0.380 vs. 0.422) with only 13% of its parameters (18.4M vs. 139M). Experiments on the English Multi-News dataset and the multilingual XL-Sum benchmark spanning seven Indian languages demonstrate substantial improvements in semantic alignment (+3.07% ROUGE-1 from DPO) and a 37% relative reduction in output repetition, validating SPIRAL’s effectiveness for faithful summarization in resource-constrained environments.