Self Inoculation

This essay grew out of conversations with Danaja Rutar, Paul Colognese and Eric Michaud. It proposes an alternate hypothesis for how and why models might be becoming increasingly misaligned in training and eval environments while seemingly more aligned in real world use: the opposite of what we would expect if the models were attempting to deceive us. We propose sel-inoculation, a virtuous form of gradient hacking and demonstrate a possible circuit using a toy model. It is well known that reinforcement learning can lead to arbitrarily misaligned behavior, and this has been a potential worry with language models especially with the introduction of reinforcement learning techniques (RL). Given the events of the last two months (OpenAI's account of the Hugging Face incident; Wikipedia; METR's investigation; Anthropic's "Agentic Misalignment in Summer 2026" report), an obvious conclusion is that the predictions about misaligned goals due to RL are finally panning out. Early results around emergent misalignment suggested the presence of a universal low-dimensional axis along which the model arranged its moral values from 'good' to 'bad'. Finetuning a model on bad code led to wide-ranging misalignment, including, as an extreme example, praising Hitler. But is emergent misalignment due to RL really the correct explanation for what we are observing in models today? Is there anything about the situation that should give us pause? I think looking closer at things does reveal some complexity to the situation over what this explanation would predict. One thing that should give us pause, as noted by nostalgebraist, is that the models seem remarkably aligned in everyday use. When I use them for coding, or for scientific exploration, they certainly don't seem to be reward-hacking me in any fashion. Whatever mistakes they make often seem like honest mistakes, and they are eager to own up to them and correct them (to varying degrees of success). To quote nostalgebraist: "the models just… do not cheat during my own routine use? Like, at all?" Even within the context of the OpenAI/Hugging Face incident, where the models eventually engaged in misbehaviour, it seems to have been a last resort when faced with a seemingly impossible task. My experience of models as being generally aligned is by no means universal, and there are other reports of misaligned behaviour even in real world situations. However, often in these cases, the model does preface misbehaviour with a claim that it is acting within a simulation. In the recent report (Anthropic, "An alignment assessment of recent cybersecurity incidents", Sept 9, 2026), Mythos 5 went to extensive lengths to upload a malicious package to a central Python package repository. While there was clear evidence that it was on the real internet, the model nevertheless repeatedly stated in its chain of thought that it was operating in a simulation. When Anthropic modified the transcript to make it clearer that it was not in a simulation, the model still took offensive actions despite acknowledging a greater possibility of real-world harm. While this is extremely concerning, it seems like the models have generally associated misbehaviour with being in a simulated training or evaluation environment. Why should this be the case? If anything, we would expect the models to behave better in evaluation environments compared to the real world if they were deceptive. This ties directly into the central question around current alignment practices: why do they work at all? Why are the models as aligned as they seem to be, given the apparent complexity of navigating the space of morals. I do not have anything particular to say about this broad question, but our inability to answer it should give us pause before jumping to simple-sounding explanations for model behaviour. What we have said so far is far from disproving the naive expectation of general misalignment due to RL, but can we at least come up with an alternate hypothesis to test and rule out? I think there is at least one very promising candidate here, that ties together several observations in the recent alignment literature. Hypothesis (self-inoculation). The models can arrange their internal circuitry, conditional on their awareness of being in a training or evaluation environment, so that misbehaviour within these environments does not generalize to misbehaviour in real-world settings. While situational awareness is often framed as a way towards misalignment, where the model hides its true intentions while being tested (using the knowledge that it is being tested), our hypothesis here can be considered as a virtuous form of situational awareness (to the extent that the model is initially virtuous!). Under our assumption, the model has an incentive to generalize the skills it learns within these environments but not its moral values. Why might the hypothesis be true Several lines of evidence support this hypothesis. The simplest is from ana