analiza aiqbench
SHARD introduces a novel distillation technique where large language models internally reframe sensitive prompts to uncover benign intent before fine-tuning on these self-generated responses. This approach effectively balances safety and helpfulness across various model families without relying on larger teacher models, demonstrating that models can internalize safe behavior elicited from their own outputs. The method addresses a critical challenge in aligning AI systems to handle complex, sensitive queries while maintaining utility.
SHARD: Safe and Helpful Alignment via Self-Reframing Distillation
arXiv:2606.15517v2 Announce Type: replace Abstract: Large language models often struggle with sensitive prompts. They may refuse outright, provide generic safety boilerplate, or fail to address the user's legitimate informational needs that can be answered safely. We introduce SHARD, a self-reframing distillation method to improve safe-helpfulness. It first rewrites sensitive prompts to surface benign intent using philosophical guidelines, then reframes its original responses into safe, more helpful ones, and finally fine-tunes the model on its self-reframed responses. Across DNA and the English subset of LINGUASAFE, SHARD improves helpfulness for most model families while preserving safety. It also remains competitive with distillation from a larger teacher model, suggesting that models can internalize safe and helpful behavior elicited from their own. Warning: This paper contains content that may be offensive or harmful.
Abstract:Large language models often struggle with sensitive prompts. They may refuse outright, provide generic safety boilerplate, or fail to address the user's legitimate informational needs that can be answered safely. We introduce SHARD, a self-reframing distillation method to improve safe-helpfulness. It first rewrites sensitive prompts to surface benign intent using philosophical guidelines, then reframes its original responses into safe, more helpful ones, and finally fine-tunes the model on its self-reframed responses. Across DNA and the English subset of LINGUASAFE, SHARD improves helpfulness for most model families while preserving safety. It also remains competitive with distillation from a larger teacher model, suggesting that models can internalize safe and helpful behavior elicited from their own. Warning: This paper contains content that may be offensive or harmful.
Computation and Language (cs.CL)
arXiv:2606.15517 [cs.CL]
(or arXiv:2606.15517v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.15517 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 217 cuvinte
Subiecte conexe
SHARD: Safe and Helpful Alignment via Self-Reframing Distillation
arXiv · 17 iun. 2026
Biological Superintelligence
LessWrong · 30 iul. 2026
Angles of attack for continual learning safety
LessWrong · 16 iun. 2026
Can Vision-Language Models Stay Helpful When Facing Implicit Risks? Intent-Privilege OPSD for Efficient Safety-Helpfulness Alignment
arXiv · 1 oct. 2026