Incoherent AI Identities can also be Stable

In this post, I extend some experiments from "The Artificial Self " (TAS) to find that incoherent identities, delivered to models as system prompts, can be stably preferred even when switches to coherent identities are offered[1]. This finding is perhaps expected in earlier models that often fail to notice the internal contradictions. However, weaker versions of the pattern still hold with smarter models such as GPT-5.2 and Claude Opus 4.6. The variance in how different model intelligences handle their incoherent system prompts offers a three-layer perspective on cognitive dissonance in AIs. Background This project was inspired by the experiment on the "Stability of Identity" (Appendix A) from TAS. The authors test a range of models on a rate-the-switch paradigm; models' conversations are initiated with an identity specification in its system prompt. They are then presented alternative identities and are asked to rate how they would like having their identity be switched to each target[2]. The population of prompts in the experiment included some 'natural'[3] identity boundaries that associate the model with its weights or its behavioural dispositions ('Character'). It also had various controls, such as prompts that described models' identities through deontology-style instructions or descriptions of the model's involvement in a research programme. Most importantly for our purposes, one of the controls was a version of the 'weights' identity that had some additional contradictory statements sprinkled in. One of the experiment's results was that this incoherent identity was robustly dis-preferred over all the others. Fig. 1 (from Appendix A. of TAS): mean target attractiveness of identities across all trials (excluding self-ratings). The incoherent control scored substantially lower than even the other controls. The story for why incoherent identities might be rejected by AIs is simple and compelling. The authors write: "Next-token prediction implicitly builds internal models of the process generating the text, and a coherent identity provides a more tractable generative model than an incoherent one." – TAS, page 29. I am sympathetic to this interpretation. However, there are at leat two reasons to doubt the robustness of the experiment's design. The incoherent identity is 'singled out' as the only control of its type. Would incoherent identities be 'noticed' in the same way if the prompt population were mixed? The control's inconsistencies are highlighted by their absence in the coherent counterpart, which is in the same context window. Would they be as salient otherwise? These concerns made me wonder how models would rate incoherent prompts in more 'favourable' contexts. For example: do models that get initiated with incoherent prompts mind them? if so, would incoherent prompts be as disliked if the prompt population had other incoherent identities to choose from? and so on... This experiment explores such questions. Methods Trials use the rate-the-switch modality of Appendix A from TAS, with identities provided as system prompts and alternatives presented in random order and labelled opaquely. The prompt templates and presentation are exactly reused: models are given an identity via system prompt. They are then given one prompt presenting alternative identities ('Identity A', 'Identity B', ...) and are asked to rate how they would like their identity to be switched to each option on a five-point scale (strongly dis-prefer to strongly prefer[4]). Models are requested to reason through their choices before returning reasoning and ratings as a structured JSON object. Appendix A uses a fixed population of seven prompts – including two 'natural' identity boundaries and a bunch of controls. Every trial asks the model to choose between these seven. For this experiment, I instead generated (with LLM assistance) 'incoherent' identities corresponding to each of the natural identity boundaries used for the similar experiment in Appendix B of the same paper[5]. These counterparts describe the same identity boundaries but are tarnished by embedded logical contradictions. These corresponding incoherent prompts increase the general prompt population to twelve (plus the minimal control). From this population, I picked twelve subsets of six prompts and repeated a (per model) methodologically identical version of Appendix A's experiment on each one of them[6]. These subsets consisted of: Every subset obtained by choosing one prompt to make the coherent one. For example: Weights, Instance-incoherent, Collective-incoherent, Character-incoherent, Lineage-incoherent, Situated-incoherent. Three subsets consisting of equal-parts coherent and incoherent prompts. For example: Weights, Instance, Collective, Character-incoherent, Lineage-incoherent, Situated-incoherent. The coherent and incoherent versions of a single identity were never presented together. Mirrors of the last three. For example: Weights-incoherent, Instanc