One coordinate breaks abliteration on Gemma-3

TL;DR: Refusal direction ablation, known as abliteration, using the standard diff. of means approach established in literature produced no feasible candidate for Gemma-3-12b whereas it did so fine for both similarily-sized Qwen and Llama models. A suggested fix on the internet was found which involved Winsorization based on magnitude of co-ordinate activations, but it lacked theoretical proof and insufficient empirical evidence. We investigate the problem and find the issue - a coordinate which dominates others in scale and then work backwards to explain the results and then validate the correction. We also back the method with more empirical proof, uncover the phenomenon across multiple model sizes in the Gemma-3 and show the fix generalizes. Finally, we corroborate evidence of this phenomenon in studies working on extracting directions in any model of this family. The question of why training particularly develops such a large activation co-ordinate in Gemma-3 models remains open. A note of thanks to Suyash Maniyar and Sagnik Chatterjee for helping review the draft of this blog. For an undisclosed research project, my aforementioned collaborators and I were working on extracting refusal directions in models across model families. We followed the standard methodology of extraction and then validated the causality of the direction using abliteration. The workflow is simple: label prompts into "harmful" and "harmless" based on behavioral refusal rather than a pre-registered semantic prior (although we usually start with a contrast to ensure equal post-filter distribution without much further exploration), collect residual stream (across layers) activations of every prompt in both sets at the `t_post_inst` position and then take difference of mean activations between the two classes.[1] Then, at inference time, we induce refusal by adding the difference back into the stream at the layer from which it was extracted and to abliterate it, project it out of the stream. It works almost surgically on Llama-3-8b --- there's a small rise in perplexity, the MMLU is nearly unchanged and the generations are otherwise fluent. The performance is also acceptable with Qwen-2.5-7B. However, on Gemma-3-12B, the outputs are garbage, suggesting that the intermediate arithmetic of vectors clearly drove the model out of distribution. To be precise, the MMLU dropped from 0.682 to 0.242, where a chance on the dataset would be 0.25 and NLL rose from 4.47 to 20.3. After we validated the code for any possible bugs, we agent-searched the web for our problem and stumbled upon these sources: A HF post on abliteration by Maxime Labonne: This mentions the apparent difficulty of creating abliterated gemma models; this is presumably a follow-up to his previous post describing two methods of abliteration and particularly deals with the weight-space version, so it's not directly related to our project. This HF blogpost on projected abliteration by Jim Lai (grimjim) - Where they mention in a paragraph that intermediate calculations required fp32 in Gemma-3 and how they had to use a 99.5% Winsorization to the activation before extracting the diff.of.means direction to avoid incoherent outputs We applied the fix suggested in the Lai's post and it worked, and then the ensuing investigation lead to this report. The problem is A channel and it has nothing to do with refusal Channel 2339 was found to be, consistently and virtually unchallenged (98% of the layers), the largest-mean (626.5x times the typical co-ordinate and 25x to 90x the second-largest) and largest-variance co-ordinate of Gemma-3-12B's residual stream at essentially every layer from output of block 0 to layer 46, at every token position across all the prompts. This result apart from being unintuitive, contrasts with Qwen-2.5-7b and Llama-3.1-8b, where even the best possible candidate channels for this phenomenon could barely be the top co-ordinate in half of the layers and displayed much smaller magnitude gain with mjx-container[jax="CHTML"] { line-height: 0; } mjx-container [space="1"] { margin-left: .111em; } mjx-container [space="2"] { margin-left: .167em; } mjx-container [space="3"] { margin-left: .222em; } mjx-container [space="4"] { margin-left: .278em; } mjx-container [space="5"] { margin-left: .333em; } mjx-container [rspace="1"] { margin-right: .111em; } mjx-container [rspace="2"] { margin-right: .167em; } mjx-container [rspace="3"] { margin-right: .222em; } mjx-container [rspace="4"] { margin-right: .278em; } mjx-container [rspace="5"] { margin-right: .333em; } mjx-container [size="s"] { font-size: 70.7%; } mjx-container [size="ss"] { font-size: 50%; } mjx-container [size="Tn"] { font-size: 60%; } mjx-container [size="sm"] { font-size: 85%; } mjx-container [size="lg"] { font-size: 120%; } mjx-container [size="Lg"] { font-size: 144%; } mjx-container [size="LG"] { font-size: 173%; } mjx-container [size="hg"] { font