Sari la conținut

Română · beta. Traducerea a fost făcută automat și este încă în curs de revizuire. Citește pagina în engleză

perspectiva aiqbenchCercetare AI

analiza aiqbench

This study challenges the prevailing use of gain ratios for measuring activation steering transfer, demonstrating that such metrics fail to reliably identify potency or efficacy when moving from single-turn chat to agent scaffolds. The authors propose replacing this ratio with a curve location difference (dEC50) as a more robust estimator across diverse model families and deployment contexts. By rigorously testing multiple deflationary accounts, the research provides critical measurement instructions for evaluating how steering effects change between different operational modes.
Sursă care acoperăarXiv
Ce este aceastaLectura unui model asupra articolului și a acoperirii conexe, scrisă pentru aiqbench. Nu sunt cuvintele publicației.

Activation Steering Transfer to Agents: One Gain Ratio Does Not Identify Potency and Efficacy

arXiv3 min de cititArticolul și perspectiva sunt în engleză.

arXiv:2607.09156v2 Announce Type: replace Abstract: Additive activation steering is calibrated in single-turn chat and then deployed inside agent scaffolds. The quantity usually reported for that move is a gain: a ratio of steered effects, T = Delta_agent / Delta_chat. We sweep eight family x arm dose-response cells over six models in both deployment contexts and show this ratio does not identify potency and efficacy. Reconstructing the published estimator in both of its forms on our own grids, its realized range contains 1 in five of five scorable cells, it moves with dose in four of five, and two cells with opposite, CI-clean potency shifts return gain intervals overlapping at a width under 0.08. Every scorable cell is an amplifier at one dose and an attenuator at another, so an amplify/attenuate taxonomy reports the dose it was read at. We replace the gain with a location: dEC50 = EC50_agent - EC50_chat, the cross-context difference in a curve location. It is two-sided across four model families (+1.013 [+0.777, +1.273] against -12.368, -10.855, -5.497 and -886.066 elsewhere) and beats a vertical rescaling at equal complexity in all five cells of a frozen audit. Three deflationary accounts (residual-norm rescaling, baseline alignment, dose transduction) are measured and rejected on sign pattern and magnitude. We report the discipline at the same volume as the result: one cell is quarantined loudly, our pre-registered forecaster was refuted out of sample and is published as refuted, a registered salvage claim produced no qualifying cell and is reported unanswered, and a census of our own register reports registered branches no code in this repository could have emitted. The consequence is a measurement instruction rather than a theorem: a transfer conclusion read at one strength does not identify what changed, because a displacement and a gain are not distinguishable from a single operating point.

Abstract:Additive activation steering is calibrated in single-turn chat and then deployed inside agent scaffolds. The quantity usually reported for that move is a gain: a ratio of steered effects, T = Delta_agent / Delta_chat. We sweep eight family x arm dose-response cells over six models in both deployment contexts and show this ratio does not identify potency and efficacy. Reconstructing the published estimator in both of its forms on our own grids, its realized range contains 1 in five of five scorable cells, it moves with dose in four of five, and two cells with opposite, CI-clean potency shifts return gain intervals overlapping at a width under 0.08. Every scorable cell is an amplifier at one dose and an attenuator at another, so an amplify/attenuate taxonomy reports the dose it was read at. We replace the gain with a location: dEC50 = EC50_agent - EC50_chat, the cross-context difference in a curve location. It is two-sided across four model families (+1.013 [+0.777, +1.273] against -12.368, -10.855, -5.497 and -886.066 elsewhere) and beats a vertical rescaling at equal complexity in all five cells of a frozen audit. Three deflationary accounts (residual-norm rescaling, baseline alignment, dose transduction) are measured and rejected on sign pattern and magnitude. We report the discipline at the same volume as the result: one cell is quarantined loudly, our pre-registered forecaster was refuted out of sample and is published as refuted, a registered salvage claim produced no qualifying cell and is reported unanswered, and a census of our own register reports registered branches no code in this repository could have emitted. The consequence is a measurement instruction rather than a theorem: a transfer conclusion read at one strength does not identify what changed, because a displacement and a gain are not distinguishable from a single operating point.

v2 changes the estimand from a gain ratio to a curve location and supersedes v1's coupling-distribution reading. 40 pages, 7 figures, 5 tables. Includes an errata section correcting v1's public record. Code and pre-registrations: this http URL

Machine Learning (cs.LG)

arXiv:2607.09156 [cs.LG]

(or arXiv:2607.09156v2 [cs.LG] for this version)

Text integral sub licența deschisă a publicației · 346 cuvinte

Subiecte conexe