analiza aiqbench
This paper proposes ES2, a fine-tuning method that widens the gap between harmful and harmless prompt embeddings, so adversarial attacks must make larger, more semantically destructive shifts to succeed. Regularization losses (an embedding anchor and a KL-divergence term) are used to limit capability loss, and the authors report some degradation on certain tasks. For practitioners, the key question is whether the robustness gains hold against stronger adaptive attacks and at production scale, which this arXiv preprint does not yet settle.
Enhancing Safety of Large Language Models via Embedding Space Separation
arXiv:2603.20206v2 Announce Type: replace-cross Abstract: Safety alignment does not fully prevent large language models (LLMs) from producing harmful content under adversarial attacks. We observe that existing attack methods typically move harmful embeddings toward regions of harmless embeddings. This motivates us to explore whether increasing the distance between harmful and harmless embeddings can improve the safety of LLMs, since greater separation requires attacks to make larger embedding shifts, which can destroy semantics. We propose Embedding Space Separation (ES2), a fine-tuning method that increases the distance between harmful and harmless embeddings. However, maximizing this distance alone can damage LLMs' capabilities. Thus, we use two regularization losses: an embedding anchor loss keeps harmless embeddings close in direction to those of a frozen base model and a KL-divergence loss keeps output distributions on harmless prompts close to those of the base model. Our experimental results demonstrate that ES2 has excellent defensive performance against various attack methods. LLMs fine-tuned with ES2 retain general capabilities comparable to those of the baseline models, though some performance degradation is observed on certain tasks. Our results indicate that ES2 is an effective way to enhance the safety of large language models, while maintaining comparable performance on general capabilities.
Abstract:Safety alignment does not fully prevent large language models (LLMs) from producing harmful content under adversarial attacks. We observe that existing attack methods typically move harmful embeddings toward regions of harmless embeddings. This motivates us to explore whether increasing the distance between harmful and harmless embeddings can improve the safety of LLMs, since greater separation requires attacks to make larger embedding shifts, which can destroy semantics. We propose Embedding Space Separation (ES2), a fine-tuning method that increases the distance between harmful and harmless embeddings. However, maximizing this distance alone can damage LLMs' capabilities. Thus, we use two regularization losses: an embedding anchor loss keeps harmless embeddings close in direction to those of a frozen base model and a KL-divergence loss keeps output distributions on harmless prompts close to those of the base model. Our experimental results demonstrate that ES2 has excellent defensive performance against various attack methods. LLMs fine-tuned with ES2 retain general capabilities comparable to those of the baseline models, though some performance degradation is observed on certain tasks. Our results indicate that ES2 is an effective way to enhance the safety of large language models, while maintaining comparable performance on general capabilities.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2603.20206 [cs.CL]
(or arXiv:2603.20206v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2603.20206 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 290 cuvinte
Subiecte conexe
Cyclical Entropy Eruption: Entropy Dynamics in Agent Reinforcement Learning
arXiv · 2 nov. 2026
Spot, Separate, and Enhance: Fully Generative Approach for Audio Mixing
arXiv · 25 sept. 2026
GenEx: A Graph-Based Representational Paradigm for SARS-CoV-2 Variant Detection via Codon Co-occurrence Networks
arXiv · 21 aug. 2026
Self-organized Layered Structures of Nitrogen-Vacancy Centers with Preferential Orientation in Heteroepitaxial Diamond Films
arXiv · 5 oct. 2026