AI breakout time: a mechanism for an enforceable AI pause

The old Iran deal aimed to keep Iran at least one year away from having a nuclear bomb. Similar controls can be used to design an enforceable AI pause. [Note: this is probably a bit underspecified in a few places. I deliberately accelerated the pace at which I wrote this piece, so as to get it out to the public as far ahead as possible of the Trump-Xi summit in late September 2026, and get people thinking about the core idea if it's useful] To pause frontier AI development, you need to define the AI capabilities threshold that must not be passed. There are a number of ways to do so. You could freeze development at the currently-existing frontier, whether freezing each company at its current level of capabilities or allowing laggards to catch up to the leader’s level while going no further. You could define a dangerous-capability threshold and prohibit further progress beyond that point. You could set a maximum allowable amount of GPUs any one entity can hold, and/or the maximal allowed amount of compute that can go into any one training run. I argue that these approaches are suboptimal, and that the threshold should be defined so as to ensure it would take a skilled team with capacious resources at least some duration of time to progress from already-available (and permitted) capabilities to gravely dangerous levels: those capabilities levels at which an AI system could either take over control from humans on its own, or be used by individual humans to enable mass-casualty attacks, pandemics, and biological or chemical warfare. Additionally, the threshold should be adaptive rather than fixed, able to be adjusted on the fly (including downward reductions). The key observation here is that any one AI model doesn’t exist in a vacuum. We are now at a stage where already-existing AI models significantly speed up the development of their successors, by empowering researchers to fine-tune systems, improve training pipelines, and investigate new avenues of research faster and more skillfully than the same researchers would be able to accomplish without this AI assistance. We are not yet at full recursive self-improvement, but every new frontier model (including models that are only deployed internally) closes a bit more of the loop, and speeds up the rate at which its own successor will be developed. Getting from 80% to 100% of the way to dangerous capabilities will be faster than getting from 60% to 80%, et cetera. Therefore, the higher you set the pause threshold, the less time it will take for rogue actors to achieve the same amount of capabilities advances above that threshold. It could very well be that 80% of the way is not in itself dangerous, but would allow a rogue actor to reach 100% of the way fast enough that we would be running the risk of not detecting them in time. The solution to this problem is to set the pause threshold not based on the level of capabilities that is itself non-dangerous, but instead based on the level of capabilities from which it would take sufficient time for a rogue actor to advance to dangerous capabilities, that we can confidently assume we will be able to catch them well in advance of reaching that threshold. There is precedent for this type of institutional design, in the Joint Comprehensive Plan of Action (JCPOA), better known as the former Iran deal. While this deal broke down due to both geopolitical and domestic political pressures, this occurred for reasons largely exogenous to this specific design mechanism - and this deal was centrally between the US and Iran, a much more difficult and untrustworthy adversary than China (the primary counterparty of any feasible AI pause treaty), with whom our relations have historically been vastly more strained. Nuclear Weapons and Powerful AI: Similarities and Differences To understand why this approach is feasible for AI, let’s take a look at the commonalities between the supply chains for nuclear weapons and for frontier AI. The research paper “Computing Power and the Governance of Artificial Intelligence” (Sastry, Heim, et al 2024) contains a useful discussion of the similarities between the raw materials needed to produce nuclear weapons, and those needed for powerful AI.[1] Both require a very specific material (enriched uranium and advanced GPUs, respectively). In both cases, the process to produce this material from its raw inputs requires cutting-edge technology whose supply chains can be restricted, monitored, and tracked. There are, of course, differences between the two processes: some which make the problem of supply-chain monitoring more tractable for advanced AI than for nuclear weapons; some which pose additional challenges for our problem and thus require additional considerations. Centrifuges, the machines used to extract highly-enriched uranium from the raw yellowcake produced by uranium ore processing, require cutting-edge materials engineering, because they must spin at 50,000-100,000 RPM in order to separate