Logo Lanfrica
  • Accueil
  • Atlas
  • Analyses
  • Documentation
  • Sign in

© 2026 Lanfrica. Tous droits réservés. Tous les droits d'auteur des ressources affichées sur le site Web Lanfrica appartiennent aux détenteurs de droits d'auteur d'origine, sauf indication contraire explicite.

Trade-off Between Harmlessness Rate and Helpfulness in DPO-aligned OPT-350M Models Across XTREME-R Language Subsets

Domaine:

natural language processing

Type de record:

paper
Créateur:
SOV
Éditeur:
Zenodo
Hôte:avatar
This research investigates the effectiveness of alignment techniques, Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and a combined SFT+DPO approach on improving the safety and helpfulness of the OPT-350M language model. Utilizing the Anthropic Helpful-Harmless RLHF dataset, we train and evaluate four models: the base OPT350M, an SFT model, a DPO model, and a model trained with both SFT and DPO. We introduce three key evaluation metrics: Harmlessness Rate (HmR), Helpfulness Rate (HpR), and a Combined Alignment Score (CAS), all derived from reward model outputs. The results Research goal: Does the trade-off between Harmlessness Rate and Helpfulness in DPO-aligned OPT-350M models vary significantly across low-resource language subsets in the XTREME-R benchmark compared to high-resource languages? Autonomous synthesis report generated by SOVEREIGN Research Kernel. Tribunal consensus score: 9.3/10. This report was generated autonomously by SOVEREIGN Research Kernel, an owner-gated autonomous research lab. The content synthesizes findings from peer-reviewed papers. Tribunal score: 9.3/10.

Visit

doi.orgzenodo.org

Tags

trade-offHarmlessnessRateHelpfulnessDPO-alignedOPT-350Mmodelsvary

Licenses

Creative Commons Attribution 4.0 Internationalhttps://creativecommons.org/licenses/by/4.0/legalcode

Similaires

Trade-off between model size and inference efficiency in zero-shot cross-lingual transfer on XTREME-RTrade-off between inference efficiency and accuracy in multilingual models with English intermediate-task training on XTREME forPerformance Gap in Cross-Lingual Retrieval Across Model Scales in XTREME-RPretraining Dense Retrieval Models on WebFAQ for Zero-Shot Cross-Lingual Recall in Low-Resource XTREME SubsetsRobustness Degradation in Intermediate-Task Training Across XTREME-R Low-Resource LanguagesScaling Intermediate-Task Training for Zero-Shot Cross-Lingual Transfer on XTREME-R Across Low-Resource Language Families

Trade-off between model size and inference efficiency in zero-shot cross-lingual transfer on XTREME-R

Pre-trained multilingual language encoders, such as multilingual BERT and XLM-R, show great potentia

Trade-off between inference efficiency and accuracy in multilingual models with English intermediate-task training on XTREME for

Intermediate-task training---fine-tuning a pretrained model on an intermediate task before fine-tuni

Performance Gap in Cross-Lingual Retrieval Across Model Scales in XTREME-R

Benefiting from transformer-based pre-trained language models, neural ranking models have made signi

Pretraining Dense Retrieval Models on WebFAQ for Zero-Shot Cross-Lingual Recall in Low-Resource XTREME Subsets

We present WebFAQ, a large-scale collection of open-domain question answering datasets derived from

Robustness Degradation in Intermediate-Task Training Across XTREME-R Low-Resource Languages

Intermediate-task training---fine-tuning a pretrained model on an intermediate task before fine-tuni

Scaling Intermediate-Task Training for Zero-Shot Cross-Lingual Transfer on XTREME-R Across Low-Resource Language Families

Intermediate-task training---fine-tuning a pretrained model on an intermediate task before fine-tuni