analiza aiqbench
This systematic study reveals a fundamental trade-off in LLM conditioning where efficient activation steering methods often severely degrade generation fluency. The research further highlights that such steering techniques are significantly less effective on instruction-tuned models compared to base counterparts, suggesting distinct optimization paths for different model types. Additionally, the findings validate cheap textual metrics as reliable proxies for expensive evaluation scores, offering practical insights for deploying robust concept injection and removal strategies.
On the Effectiveness-Fluency Trade-Off in LLM Conditioning: A Systematic Study
Controlling the output of Large Language Models (LLMs) is a central challenge for their reliable deployment, yet a clear understanding of the involved trade-offs remains elusive. Current approaches to conditioning are often evaluated with a narrow focus on their effectiveness at injecting or removing a target concept, neglecting generation quality. We systematically investigate a range of conditioning methods in both injection and removal scenarios. We find that efficient steering methods frequently achieve conditioning at a steep cost to fluency. Furthermore, we identify a critical yet…
Controlling the output of Large Language Models (LLMs) is a central challenge for their reliable deployment, yet a clear understanding of the involved trade-offs remains elusive. Current approaches to conditioning are often evaluated with a narrow focus on their effectiveness at injecting or removing a target concept, neglecting generation quality. We systematically investigate a range of conditioning methods in both injection and removal scenarios. We find that efficient steering methods frequently achieve conditioning at a steep cost to fluency. Furthermore, we identify a critical yet previously overlooked interaction with the training paradigm: activation steering methods are far less effective on instruction-tuned models than on their base counterparts. Simple prompting and full-fledged supervised fine-tuning, on the other hand, are viable options for concept injection, but are not as good at concept removal. Finally, cheaply computed textual metrics highly correlate to costly LLM-as-judge scores, and provide insights on the behavior of conditioning methods.
Related readings and updates.
Activation steering has emerged as a powerful method for guiding the behavior of generative models towards desired outcomes such as toxicity mitigation. However, most existing methods apply interventions uniformly across all inputs, degrading model performance when steering is unnecessary. We introduce Dynamically Scaled Activation Steering (DSAS), a method-agnostic steering framework that decouples when to steer from how to steer. DSAS…
In the context of a voice assistant system, steering refers to the phenomenon in which a user issues a follow-up command attempting to direct or clarify a previous turn. We propose STEER, a steering detection model that predicts whether a follow-up turn is a user’s attempt to steer the previous command. Constructing a training dataset for steering use cases poses challenges due to the cold-start problem. To overcome this, we…
Text integral sub licența deschisă a publicației · 285 cuvinte
Subiecte conexe
On The Effectiveness-Fluency Trade-Off In LLM Conditioning: A Systematic Study
arXiv · 11 iun. 2026
On The Effectiveness-Fluency Trade-Off In LLM Conditioning: A Systematic Study
arXiv · 14 sept. 2026
Angles of attack for continual learning safety
LessWrong · 16 iun. 2026
Why do models task game?
AI Alignment Forum · 6 aug. 2026