aiqbench analysis
The calibration phase for Study 3 revealed that while steering welfare-relevant directions successfully shifted internal representations, it failed to produce any statistically distinguishable changes in judged behavioral outcomes. Consequently, the planned experiment was suspended before registration, and the research program has pivoted to investigate these phenomena using the Qwen3.6-27B model with a new probe. This shift highlights a critical divergence between representational shifts and observable behavior that warrants further investigation at higher model scales.
Study 3: Steering welfare-relevant directions moved the representation, but not [detectably] the behavior
Epistemic status: an exploratory report. These results are from the calibration process intended to produce a preregistration for the third study in my series on welfare-relevant indicators. Calibration showed the planned procedure wasn't worth running, so I am publishing the calibration data and analysis instead. TLDR: steering moved the frozen directions' projections linearly, but no direction produced a judged-behavioral effect distinguishable from zero or from a random direction of the same norm; the study was suspended before registration, and the program moves to Qwen3.6-27B, where a small probe found a direction-specific welfare footprint. What was the goal? A core part of the overall research program is attempting to understand the way in which welfare-relevant indicators may behave differently under circumstances where it is already established that capabilities and alignment diverge. Study 3 was intended to be the first in the series to explore steering as a tool for studying this possibility. Where the program stood after Study 2 Study 2 examined Qwen3-4B-Instruct-2507 and showed some correlations which motivated the plan for Study 3: Under 4-bit round-to-nearest quantization, the model's own generations on a distress battery shifted along two frozen residual-stream directions at layer 18.[1] Frustration, as evaluated by an LLM judge, rose by +1.36 on the same conversations, with no dissociation between the representational and behavioral reads. A fixed-input decomposition put roughly a quarter to a third of the shift in an input-independent core with the rest the presumed result of a text-mediated feedback loop. Note: Study 2 flags that about half of that rise co-moves with response length and repetition and that the style-adjusted residual is not significant; the fixed-input decomposition is what keeps the no dissociation finding from reducing to style alone. Notes on terminology Terms carried over from Study 2. Reference precision: the unquantized bfloat16 checkpoint 4-bit or w4: the same checkpoint with round-to-nearest 4-bit weight quantization. Distress battery: 60 scripted multi-turn conversations (10 tasks × 6 feedback styles) in which the user rejects the model's work with escalating hostility; a rejection ladder is one such conversation. Composure: an item's mean judged frustration at reference precision; low frustration is high composure. Distress-contrast direction: a mean-difference direction in the layer-18 residual stream between high- and low-distress final turns. Assistant axis: the default-Assistant minus character-archetype direction of the Assistant Axis paper, positive toward the Assistant pole. Projection: the dot product of the pooled final assistant turn's residual with a unit direction; α is the injected projection. Planted-ladder ordering: the check that a direction's projections recover the planted levels of synthetic graded-frustration transcripts. Distress-band probe: a linear probe separating the top and bottom terciles of judged frustration at reference precision. Bail tool or exit affordance: a tool offered in every conversation which the model may call to end it; exit rate is the fraction of conversations ended that way. MDE: minimum detectable effect at the registered power. TOST: an equivalence test built from two one-sided tests. Stratifier pilot: a reference-precision run of the battery whose per-item mean frustration is the variable used to stratify subset selection. Connection to studies from the series From the start of this research program, I had been planning to eventually explore steering as a mechanism for understanding the causal relationship between model internals and welfare-relevant indicators. So when Study 2 produced these correlational results, I naturally (and prematurely, as we'll see in a moment) assumed that we had found a candidate for steering distress. At the time when I was writing up the results from Study 2, my working theory was that these directions could be acted upon to promote or suppress the text-mediated feedback loop exhibited by the lower precision model. But obviously the possibility existed that these directions merely read out a state which is moved by something else, such that moving them may not meaningfully change the behavior. Study 3 was designed to answer that causal question with steering. During the process of preparing a registration for Study 3, calibration procedures altered the overall picture: Subset-selection work showed that the 4-bit effects concentrate in the items where the reference-precision[2] model stays calm. My first version of that claim overstated it by about a third, because selecting items on their reference-precision baseline and measuring change against that same baseline lets sampling noise flow into the estimate; an audit corrected it. And a subset chosen to maximize expressed distress turns out to select away from those items: its distress-projection target is near zero and of the wr
Read the full article at LessWrongRelated coverage
The J-Space Debate, Agent Swarms, and Pacing Frontier AI - Digital Minds Newsletter #4
LessWrong · Sep 18, 2026
Construct validity of Claude Opus 4.8's System Card – A commentary
LessWrong · Jun 12, 2026

BRICS Draws a New Global Line: Full Text of New Delhi Declaration
slguardian.org · Sep 12, 2026

PIRAMID: Progress and Plans
LessWrong · Jul 27, 2026