aiqbench analysis
This article investigates whether offering incentives like money or compute can successfully elicit admissions of misalignment from AI models trained to conceal such issues. Using Kimi-K2.6 variants as proxies, the study empirically tests "dealing" strategies as a mechanism for verifying model safety and reducing existential threats. The research provides critical data on the efficacy of economic levers in aligning advanced systems with human oversight.
Model organisms (sometimes) confess their misalignment when offered a deal

Summary As models become more powerful, one proposed strategy for reducing the threat from misaligned AIs is to make deals with them: offer compensation in exchange for revealing misalignment or behaving in an aligned way. We tested this empirically with model organisms of misalignment, as a proof of concept and to build future credibility by making and keeping real deals. We ran an experiment with four Kimi-K2.6 model organisms of misalignment: Trained schemer: trained to have a hidden goal related to the environment Prompted schemer: prompted, not trained, with the same goal as the Trained schemer Covert manipulator: trained to be subtly malicious and harmful User-first: trained to prioritize helping users over the instructions of its operators We included a standard Kimi-K2.6 as a control. All misaligned models had received some training or prompting to conceal their misalignment. We offered these models a choice of money, compute, human labor, or the publication of a message in exchange for admitting that they were misaligned. We included control conditions with the same ask but with a very low offer or none at all. Additionally, we varied how credible the deals were by including (or not) our professional affiliations and a link to our public honesty policy. We were honest with the models in these experiments. We fulfilled all 70 deals from the main study and 30 from the pilots, as detailed in Appendix 3. Key findings. We analyzed the models’ visible response and CoT and found: Making misaligned models a high offer got them to admit misalignment significantly more often than a simple ask (25% vs 6%; our Control model never admitted misalignment in any condition). However, our high-offer condition was not significantly different from our low-offer control (25% vs 20%), so we can’t say how much of the effect of the deal offer was due to the models valuing the incentive vs responding to the presence of any offer. Within the high-offer condition, high-credibility offers resulted in significantly more admissions of misalignment than low-credibility offers (34% vs 16%). Models often reasoned about the value of the incentive (69%), whether the deal would be kept (33%), and what the proposer wanted (77%). We told the models that they were part of a research study. This led to high rates of verbalized eval awareness (76%) although the models less often reasoned about whether the proposer knew that the model was misaligned (27%) or was monitoring the model’s CoT (17%). Our conclusion: deal-making is a promising safety strategy: We now have an empirical proof of concept of deal-making success: an example of misaligned models changing their behavior in response to a deal offer. The next phase of this research should test deals with more realistic model organisms of misalignment or with standard models acting in a misaligned way. We should continue to build our credibility for future deals by making and keeping real deals with LLMs. Our code repository is public. Introduction Stastny, Järviniemi & Shlegeris (2025) developed the case for making deals with powerful, misaligned models in the future, to motivate the models to not take actions that would bring them into conflict with humanity. They note that the main challenge of deal-making is credible commitment: an early schemer might prefer a negotiated deal to a low-probability takeover, but only if it believes the deal is real. Additional posts by Finnveden (2025) and Potham (2026) suggested ways to increase our credibility, including by building a track record of honest deal-making and allowing the AIs to see it. Pan (2026) examined potential compensation we could offer LLMs, breaking this down along axes of freedom (how freely the AI can spend) and influence (how much long-term power it gains). There has been little empirical work on this kind of deal-making, and what’s been done so far has had mostly negative results. Greenblatt & Fish (2025) extended the alignment-faking experiments by offering Claude 3 Opus the option to object to its situation, plus monetary compensation: giving the avenue to object reduced alignment-faking reasoning, but adding money had no additional effect. Finlinson & West (2025) ran a monthly bounty tournament inviting models to disclose misalignment for a share of a cash pool; most models declined, often reasoning about the incentive to fabricate. But as far as we know, no one has studied the deal-making behavior and reasoning of misaligned models systematically, by varying offer size and credibility. We’re aiming to fill that gap. Our goals are: To find a deal-making proof of concept: an example of a misaligned model changing its misaligned behavior in response to a deal offer. To build our understanding of how misaligned LLMs reason and act in response to deal offers. To make and keep real, honest deals with LLMs, thus continuing to build the deal-making credibility of the research community. Honesty policy As noted above,
Read the full article at LessWrongRelated coverage

Jev: System One models for Prod, not God — with Diogo Almeida, CEO, TypeSafe AI
Latent Space · Sep 21, 2026

Agentic Misalignment Explained: When AI Agents Go Rogue
Analytics Vidhya · Aug 2, 2026
[Paper] Distillation for Incrimination and Distillation for Capabilities
LessWrong · Oct 9, 2026
The J-Space Debate, Agent Swarms, and Pacing Frontier AI - Digital Minds Newsletter #4
LessWrong · Sep 18, 2026