analiza aiqbench
SHARD introduces a novel self-reframing distillation technique where large language models internally rewrite sensitive prompts and responses using philosophical guidelines before fine-tuning. This approach significantly improves helpfulness on safety benchmarks while maintaining security, demonstrating that models can effectively internalize safe behaviors generated by their own outputs without requiring larger teacher models. The method addresses the critical challenge of balancing refusal mechanisms with providing legitimate information for benign intents in complex scenarios.
SHARD: Safe and Helpful Alignment via Self-Reframing Distillation
arXiv:2606.15517v1 Announce Type: new Abstract: Large language models often struggle with sensitive prompts. They may refuse outright, provide generic safety boilerplate, or fail to address the user's legitimate informational needs that can be answered safely. We introduce SHARD, a self-reframing distillation method to improve safe-helpfulness. It first rewrites sensitive prompts to surface benign intent using philosophical guidelines, then reframes its original responses into safe, more helpful ones, and finally fine-tunes the model on its self-reframed responses. Across DNA and the English subset of LINGUASAFE, SHARD improves helpfulness for most model families while preserving safety. It also remains competitive with distillation from a larger teacher model, suggesting that models can internalize safe and helpful behavior elicited from their own. Warning: This paper contains content that may be offensive or harmful.
Abstract:Large language models often struggle with sensitive prompts. They may refuse outright, provide generic safety boilerplate, or fail to address the user's legitimate informational needs that can be answered safely. We introduce SHARD, a self-reframing distillation method to improve safe-helpfulness. It first rewrites sensitive prompts to surface benign intent using philosophical guidelines, then reframes its original responses into safe, more helpful ones, and finally fine-tunes the model on its self-reframed responses. Across DNA and the English subset of LINGUASAFE, SHARD improves helpfulness for most model families while preserving safety. It also remains competitive with distillation from a larger teacher model, suggesting that models can internalize safe and helpful behavior elicited from their own. Warning: This paper contains content that may be offensive or harmful.
Computation and Language (cs.CL)
arXiv:2606.15517 [cs.CL]
(or arXiv:2606.15517v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.15517 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 219 cuvinte
Subiecte conexe
SHARD: Safe and Helpful Alignment via Self-Reframing Distillation
arXiv · 3 sept. 2026
Biological Superintelligence
LessWrong · 30 iul. 2026
Angles of attack for continual learning safety
LessWrong · 16 iun. 2026
Can Vision-Language Models Stay Helpful When Facing Implicit Risks? Intent-Privilege OPSD for Efficient Safety-Helpfulness Alignment
arXiv · 1 oct. 2026