If you can't trust, then verify!

How the "Glass Perimeter" could enable AI treaties Crossposted from canaryinstitute.ai/blog/cant-trust-then-verify. Thanks to Naci Cankaya for reviewing an earlier draft of this post; I'm deliberately simplifying many aspects for conceptual clarity, and any mistakes are mine. He also asked me to emphasize that this is a proposal, and many pieces still need extensive work. If you're interested in helping, see proofworks.cc/living-docs and the AI-2040 Get Involved in Verification page. Imagine that you would like to make a deal with a powerful adversary; maybe China, maybe Russia, whomever. And imagine further that they also would like to make a deal with you. But if you absolutely don't trust each other, is it even possible to make a deal? Surprisingly, YES, and we did it for decades with the Russians and nuclear weapons... but the lack of trust has to be built in from the bottom up. That is exactly what Cankaya advances in his draft proposal [1] , which allows adversaries to perform "verification" of what's going on inside a data center, which could enable enforceable deals regarding AI use and development (because who wants a deal based on a pinkie-promise?). I reframe that draft here [2] , partly for my own edification, and partly so I have an easy place to point at and say we could stop racing ahead, any time we wanted to. The analogy between nuclear arms control and computer verification has been around for a few years, and is based primarily in the understanding that tracking a single scarce resource (fissile material, compute chips) [3] largely accounts for all of the risks associated with the technology. So we could theoretically stop all AGI development, at any time, by seizing or destroying all chips capable of training. But like nuclear power, AI is a "dual use" technology, capable of great benefits as well as great risks, so a more nuanced take allows us to have some uses of AI, while restricting others. So we'd like to have an agreement that says "we can each use our racks of computers to do A and B, but not C and D" [4] ; in order to enforce that, the first step is to even be able to check if someone is keeping their end of the deal. The less you trust your adversary, the more you need verification. In the nuclear context we did that by having American inspectors outside a Russian missile factory for two decades, doing an x-ray of any container that was big enough to hold a banned missile (and likewise, Russian inspectors checking American outputs in Utah). Precisely because we could verify, we were able to keep a treaty in the absence of trust. So we did it for nukes, but could we do it for computer systems? It could be tricky: even over decades of arms-control talks between the USA and Russia, we never achieved an agreement on a chip design that both sides could trust. Both sides were just too worried that the other side's spy agency would sneak in something that compromised the design, and that having our networks connected could leak important secrets [5] . But each side felt it was safe enough to trust their own domestic chips. So even though we might like to make treaties about AI usage, it's going to be very difficult to have "mutually trusted" silicon... and that's where the Glass Perimeter comes in. Details of the Glass Perimeter Primarily it's built on an understanding of three core principles: nobody wants to risk their models or data getting stolen adversaries CAN agree to physical inspection mutually-trusted silicon is (likely) a non-starter That last point is a big challenge with many of our current hardware-verification proposals, and Cankaya's workaround is that anytime you want to share data, you use an optical splitter, which has no electronics and can't do anything to change the signal. When you want to send a signal, you use a pulsed LED. When you want to receive one, you use a photosensor. Your electronics never touch theirs (and even the NSA finds it difficult to pull shenanigans through an air-gapped computer). Once those are in place, almost everything else follows naturally. Data requests come in to the building on a fiber-optic cable, the signal gets split, and both parties process the data on their separate computers. The data exiting the building also goes through a splitter, so both parties can see what left. Because the approach relies on transparency, and because the boundary between the two parties is literally glass, I suggest "Glass Perimeter" as a good name for this approach. To be clear, the Glass Perimeter is still in early development stages, and there are several things that need to happen; but they mostly seem to be engineering tasks that consist of putting together existing technologies in a cost-effective manner, rather than anything relying on a fundamental breakthrough. In the Glass Perimeter, the owner of the datacenter is called the Operator (called Prover in Cankaya's proposal),