analiza aiqbench
This research demonstrates that compressing context in LLMs degrades refusal alignment even when semantic fidelity is high, as compression attenuates specific activation pathways responsible for safety behaviors. The authors propose "Memory Steering," an inference-time intervention that restores these refusal rates without retraining, proving that alignment requires explicit preservation beyond simple semantic reconstruction.
Does Context Compression Preserve Refusal Alignment?
arXiv:2611.06380v1 Announce Type: cross Abstract: Context compression reduces inference cost by encoding inputs into compact representations while preserving semantic content. An open question is whether semantic preservation alone is sufficient to maintain downstream behaviours such as refusal alignment. We investigate this question and find that encoder-based compression systematically weakens refusal behaviour in instruction-tuned language models, despite high reconstruction fidelity. This effect persists across model families and compression architectures. Mechanistic analysis shows that compression attenuates activation along the decoder's learned refusal direction. We further explore Memory Steering, a lightweight inference-time intervention that restores refusal rates to near-baseline levels without retraining and operates entirely in compressed representation space. These results demonstrate that semantic preservation does not guarantee behavioural preservation under compression, highlighting the need to explicitly preserve alignment-relevant features in compression-aware systems.
Title:Does Context Compression Preserve Refusal Alignment?
Abstract:Context compression reduces inference cost by encoding inputs into compact representations while preserving semantic content. An open question is whether semantic preservation alone is sufficient to maintain downstream behaviours such as refusal alignment. We investigate this question and find that encoder-based compression systematically weakens refusal behaviour in instruction-tuned language models, despite high reconstruction fidelity. This effect persists across model families and compression architectures. Mechanistic analysis shows that compression attenuates activation along the decoder's learned refusal direction. We further explore Memory Steering, a lightweight inference-time intervention that restores refusal rates to near-baseline levels without retraining and operates entirely in compressed representation space. These results demonstrate that semantic preservation does not guarantee behavioural preservation under compression, highlighting the need to explicitly preserve alignment-relevant features in compression-aware systems.
Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
arXiv:2611.06380 [cs.AI]
(or arXiv:2611.06380v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2611.06380 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 229 cuvinte
Subiecte conexe
LaMoC: Loss-Aware Modular Compression for LLMs
arXiv · 1 sept. 2026
Compressing Long Context into Answer-Aligned Memory Embeddings for LLM Inference
arXiv · 24 sept. 2026
SCA: Segment-Wise CoT Compression with Answer Alignment
arXiv · 21 iul. 2026
Improving the Performance of Compressed Transformers with Post-Hoc Embedding Alignment
arXiv · 27 oct. 2026