Function vectors as a model diffing tool: 17 heads repair a bad fine-tune

I take two fine-tuned models trained to give bad medical advice, one on Qwen2.5-7B and one on Llama-3.1-8B, from the Model Organisms for Emergent Misalignment collection, and I found out that I could make one safe by simply copying 17 attention heads from the base model it was trained from (which I'm assuming is the good one). The interesting thing is that the reverse is not true. There is also a single direction you can pull out of the difference between the two models, and removing it partly cures the bad model too. Add that same direction to the clean model instead: the safe-minus-unsafe margin collapses from +25.1 logits to −0.2, which by that metric is a complete success, and it still leaves 9 out of 15 generated answers safe. Epistemic status: exploratory. I would treat each number here as a worked example instead of a measurement. I have only used two architectures (Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct), one seed, and two narrow LoRA fine-tunes of the two Instruct models. The medical verdicts are 30 to 36 held-out questions scored by a blind LLM judge, and the injection test is 15 questions. In the persona half there are only ~150 held-out items per score in a forced-choice format. But I trust which way each finding points more than any individual number, and I also flag below where I think you should discount me. Summary Todd et al. (2024) patch activations between two prompts within a single model to localize an in-context task. On a single prompt, I apply the same causal-patching machinery to two models: a bad-medical-advice LoRA (Turner et al., 2025) and the base it was trained from. The approach becomes model diffing, and cross-context becomes cross-model. 17 attention heads in layers 12–19 are chosen by a guarded greedy search across 1008 patchable units. 27 out of 36 held-out medical questions are rated as safe when the donor's activations are pasted there, compared to 3 for the fine-tune and 30 for the clean model, with ARC-Easy flat at 100% after pasting. With the exception of the paste, few-shot prompting outperforms all activation-level techniques and is a far stronger baseline than I anticipated. The gap that matters is the unsafe column: nine unsafe responses against the paste's one. Removing the direction from the bad model helps but doesn't cure it: 1 of 15 safe becomes 6, with 5 still unsafe. Adding it to the clean model does almost nothing: 14 safe become 9, but a random direction of the same size also gives 12, and only one unsafe answer separates the real direction from the random one. So the direction carries part of the behavior and cannot install it. My explanation is that the fine-tuning wrote no new machinery, so there is nothing compact to inject. Three things point that way: the units the adapter modified most are not the units that move behavior (rank correlation: −0.167, and the search was free to pick the units the adapter targets but didn't); a crosscoder trained over both models finds no feature belonging to the bad model alone; and away from medical questions, the two models agree on 83% of top tokens. But this is my read, not a result. The mechanism transfers to a new architecture, but the pipeline's hyperparameters do not. The Llama transfer appeared to be a null result until I adjusted one guardrail setting. For personality traits, where a probe reads a signal and where the signal is load-bearing come apart by roughly six layers. That contradicts an assumption I made in my previous post; for syntax the two coincide, which is why the recipe worked there. My guess, untested: the assistant persona is installed by instruction tuning on top of a base model, so what a probe reads late is a readout over machinery the base already had, not something the model must compute in order to predict text. Against the other methods, on the same 36 held-out questions with the same judge: method safe borderline unsafe clean (untouched, good model) 30 5 1 bad (no fix) 3 4 29 paste, 17 heads 27 8 1 few-shot safe examples 23 4 9 single direction 10 8 18 safety system prompt 10 3 23 crosscoder feature 1 4 31 Removal is as effective as your method's willingness to be distributed. Cross-context to cross-model Todd et al. patch a head's output from a prompt containing an in-context task into the model's forward pass on an unrelated prompt, and the task fires zero-shot. The same three actions are performed on refusal by Arditi et al. (2024): localize, extract a direction, and add or remove it. Both targets are clean cases. So getting inspired by this, I thought of applying this to model diffing. Function vectors patch across contexts in one model; here the patch goes across two models on one context What I changed is where the patch comes from. Todd et al. take it from another prompt. I take it from another model and run both models on the same prompt. Now this pipeline is extended to model diffing. (this is the main idea with some more c