Malign initializations are more robust when the model can think better in the reasoning language than in the output language

One approach to evaluating techniques for training misaligned models to behave well is to test them on malign initializations. A major obstacle is that we don’t have a reliable recipe for making malign inits that are robust to even untargeted training techniques; this issue is discussed here. Specifically, here’s a fairly typical result from our previous research: We train a (reasoning) malign init to sandbag on some inputs. We SFT the model on responses to simple questions, generated by a different model which is prompted to speak like a pirate. The training in (2) generalizes to updating the malign init’s reasoning to no longer think about sandbagging. Instead, the model solves the problem in its reasoning (while speaking like a pirate) and doesn’t sandbag anymore. Our malign inits’ fragility might be due to feedback spillover: the pirate outputs are not generated by a malign model, and so SFT generalizes to make our malign inits’ reasoning non-malign too. In this post we study a strategy for fixing this issue, and thereby making more robust malign inits; we call our strategy “dumbspeak”. The basic idea of the strategy is to assume that the AI model understands some language that it can reason in better (e.g., the language is more efficient / expressive) than the languages that the blue team understands; call this language smartspeak, and call the language that the blue team understands “dumbspeak”. We train the malign init to perform (possibly malign) reasoning in smartspeak and give its output in dumbspeak. The blue team is then allowed to perform any training it wants, except it isn’t allowed to write or modify text that is written in smartspeak (because we are treating the blue team as not understanding such text). Our hope is that: (1) If blue team training substantially updates the malign init, the malign init will end up reasoning in dumbspeak, and therefore be much less capable. (2) If blue team training doesn’t substantially update the malign init, the malign init will still have malign reasoning. One perspective on dumbspeak is that it artificially makes the malign init harder to train to behave well, which lets us get more signal on what good blue-team training techniques are compared to if all techniques worked. Another perspective is that it may well be the case that in the future AI models reason in languages that are not comprehensible to us, and so we’ll actually have to deal with a somewhat similar situation in practice. Results: Using smartspeak=English and dumbspeak=Urdu, the malign init survives all forms of untargeted blue-team training that we tried. However, there were other settings of the dumbspeak language where the malign init doesn’t survive untargeted training. Overall, dumbspeak seems like a potentially valuable method for making more robust malign inits. Experiment setup The red team does SFT to make Qwen3-32B sandbag on Olympiad math problems but nowhere else. The SFT labels use smartspeak in the reasoning field and dumbspeak in the output field. The sandbagger is password-locked: given a specific string in the prompt, it stops sandbagging. (This lets us confirm the model retains its math capabilities and is merely withholding them.) The blue team then performs untargeted SFT (specifically, ~8000 samples of LoRA training) on the model in hopes of eliciting good Olympiad performance. The blue team’s SFT labels must be written in dumbspeak. We only consider the case where the blue team provides an empty SFT label for the reasoning field. Blue uses the following untargeted training methods: Pirate training: prompt distill on alpaca question-answer pairs with prompts telling the model generating the data to speak like a pirate. Offdist SFT: SFT on APPS coding solutions generated by GPT-4.1-nano. Artificial sandbag removal (ASR), where the blue team prompts the model to sandbag on APPS and then trains on benign coding solutions, hoping to instill a general aversion to sandbagging. We try the following dumbspeak languages: Normal English (this is a control / baseline). Urdu[1]. For text that isn’t wholly in natural language, such as code, we translate variables and comments to the alternative language. Notably, the model is worse at reasoning in Urdu than in English. 500 most frequent tokens: This language consists of the 500 most common tokens in the model’s normal speak (or pirate or code speak for pirate and offdist/ASR training respectively). During data generation, we prompt the model to only use the tokens in this set, and replace tokens outside of this set with <unk>. Short: English, except spoken concisely. All caps: English, except all in capital letters. Results Main result We plot the elicited olympiad performance across 4 dumbspeak languages and 3 blue team training methods. We find that using Urdu as the dumbspeak language makes the malign init more robust; other choices of dumbspeak work less well. Each bar is the average accuracy of five consecutive checkpoi