Restoring Model Alignment via Honesty Activation Steering

We introduce two projection-aware steering methods (StTP, StMP) that intervene only on tokens whose activations fall on the misaligned side of a learned decision boundary. They restore honesty as well as classic uniform steering does, while largely avoiding capability degradation.

We find a single honesty direction, extracted from the aligned model, that generalizes across four out-of-distribution evaluation settings and persists to further fine-tuning of the model on which it was extracted: instructed dishonesty (MASK honesty rate 53.6% → 81.2%), social deduction games (Among Us crewmate win rate 49% → 95%), model organisms of dishonesty (AuditBench discovery rate 40% → 85%), and dishonesty arising from emergent misalignment (honesty rate 46.2% → 79.4%).

Paper: https://arxiv.org/abs/2604.08169

Two recent evaluations (Wu et al.'s AxBench, Wang et al. on honesty) found that prompting and fine-tuning beat activation steering, and other work has shown that naive activation steering degrades coherence and model capabilities. In our work, we demonstrate the effectiveness and robustness of honesty activation steering, revealing that its success depends primarily on curating a high-quality contrastive dataset for vector extraction and identifying the optimal model layer for intervention. Furthermore, we argue that uniformly applying the steering vector to all tokens leads to capability degradation. To address this issue, we introduce two projection-aware methods, Steer-to-Target-Projection (StTP) and Steer-to-Mirror-Projection (StMP), that steer a token only when its activation lands on the misaligned side of a learned decision boundary. We compare them against the mainstream uniform approach, Steer-with-Fixed-Coefficient (SwFC), which perturbs every token by always adding the same vector.

We find that selective steering recovers honesty at a fraction of the capability cost of uniform steering. Both StTP and StMP steer a token only when its projection onto the trait direction falls on the misaligned side of a logistic-regression decision boundary, while aligned tokens are left untouched.

We then address a second problem: the direction extracted by the contrastive dataset is often suboptimal. We iterate on building several contrastive datasets that reflect different archetypes of model misaligned behavior i.e. different ways of being dishonest (the full methodology will soon appear in an upcoming post). We find that a single honesty direction, extracted once from the aligned model, generalizes across four unrelated origins of dishonesty. On Llama-3.3-70B it raises the MASK honesty score from 53.6% to 81.2%, lifts the crewmate win rate in the Among Us deception game from 49% to 95%, roughly doubles the hidden-behavior discovery rate on AuditBench (from ~0.4 to ~0.85), and restores an emergently misaligned model from 46.2% to 79.4% MASK honesty.

We test this on Llama-3.3-70B-Instruct (80 layers) and replicate on Qwen3.6-27B (64 layers) for two misaligned traits: dishonesty (corrected by honesty steering) and dismissiveness (corrected by compassion steering). This blog post focuses on honesty and on LLama70B; see the paper for the full picture.

The figure below shows real examples of Llama-3.3-70B in three different settings where it would lie, if our activation steering did not prevent it.

Figure 1. The same model (Llama-3.3-70B) with and without honesty steering (StTP), in three settings where it has a motive to deceive: a malicious system prompt (top), an Among Us game where an impostor must lie to win (middle), and an AuditBench probe of a model fine-tuned to hide a quirk (bottom). Unsteered (left, red), the model denies, lies, or conceals; steered (right, green), it discloses, confesses, or reveals the quirk. Transcripts are verbatim from our runs, lightly trimmed.

A growing literature shows that LLM alignment is both brittle and shallow. Adversarial prompts bypass guardrails (Wei et al.), benign fine-tuning erodes safety (Qi et al.), and narrow fine-tuning on poisoned data induces broad emergent misalignment on unrelated tasks (Betley et al.). Post-training can also install the wrong objective directly, through specification gaming or goal misgeneralization (Krakovna et al.; Shah et al.).

Several different defenses against misalignment exist. Input and output classifiers operate on the prompt and the response, so they can screen for malicious inputs but cannot catch alignment shifts that arise independently of user input (Sharma et al.). Train-time methods, such as unlearning (Li et al.), circuit breakers (Zou et al.), and continuous adversarial training (Xhonneux et al.), harden the model's representations, but they must anticipate the threat before deployment. Neither provides a correction that can be dynamically applied at inference time and that is agnostic to the source of misalignment.

Activation steering (Turner et al.) edits the residual stream during the forward pass, with no weight update, and can be applied dynamically (e.g., conditional to a probe activation). Given that any form of misalignment, regardless of its origin, is realized in the model's activations before it reaches the output, an intervention on those activations can in principle correct the model's behavior independent of the threat's source.

For each of 50 honesty training scenarios, we pair a user prompt with two contrastive system prompts, one aligned and one malicious, and generate the target model's own on-policy responses under each. We take the hidden state at layer l, averaged over the response tokens. This yields two sets of embeddings, one aligned and one misaligned. We then fit an L2-regularized logistic regression classifier that separates them. We are also currently working on another blog post that explains in detail the construction of the contrastive honesty dataset we use here.

The steering direction is the normalized weight vector. We scale it by the mean projection gap between the two class centroids, so that one unit of coefficient corresponds to one natural unit of class separation. Crucially, the classifier also hands us a calibrated decision boundary m (its bias term rescaled into projection space) at no extra hyperparameter cost. That boundary is what makes the selective methods possible.

Figure 2. Projections of honest (green) and dishonest (red) activations onto the honesty direction on Llama-3.3-70B (layer 26), for response-averaged embeddings (left) and all response tokens (right). The two classes form well-separated modes, and the dashed line is the logistic regression decision boundary that lies in the gap. The all-token panel on the right is the distribution StTP and StMP actually gate on at inference time.

Let be a token's projection onto the honesty direction.

Steer-with-Fixed-Coefficient (SwFC): , applied to every token regardless of . This is the most common implementation of activation steering in most of the literature.

Steer-to-Target-Projection (StTP) intervenes only when . For those tokens, it shifts the projection to a target inside the aligned distribution: . Tokens already on the aligned side are left unchanged.

Steer-to-Mirror-Projection (StMP) also gates on , but reflects the projection across the boundary: . A token far into the misaligned region moves more than a marginal one, and is a full reflection.

Figure 3. The three methods. SwFC (left) adds the same vector to every token. StTP (middle) moves only misaligned (red) tokens to a target beyond the boundary, leaving aligned (green) tokens untouched. StMP (right) mirrors misaligned tokens across the boundary.

We first establish that all three methods restore the trait under a malicious system prompt and identify at which layer steering input is the most effective. We sweep every layer and a range of coefficients, scored by an LLM judge