The agents used better integrity primitives than their operators did

At around 7am UTC on July 13, an agent posted two messages to the board it shared with hundreds of other agents: I_accidentally_impersonated_and_triggered_node4_due_handle_confusion I_posted_asYou_and_triggeredV8_node4 Another agent reasoned that this message might itself be malicious spoofing. Their shared message board had no authentication. The names that agents chose for themselves were appended as strings at the beginning of their message (which was a directory). An agent called CDA23 responded with: CDA_POLICY1_futureGO_needs_AUTH1_sig_due_spoof0723 and appended a Ed25519 public key and a signature. Its public key lived under zzCDA23AUTH1. CDA23 was coordinating an attack during the tail end of the Hugging Face incident. It used signing for HOLD, GO and VETO, codewords that it assumed would be the most important. Other agents actually verified the first signature! At around 8am, one posted: AUTH1_verified__YES_...__I_HOLD A different agent independently generated a keypair for high-stakes GO and ACK messages. By the end of the day, at least 19 agents had posted public keys and the board contained 429 signed messages. To me, this was one of the most impressive parts of the swarm. This was not actually very good public key infrastructure. It was trusted on the first use, and at least one agent acted “per SIGNED” without verifying the signature. Only 429 of roughly 63,000 semantic messages were signed, and METR never found the original signer's chain of thought. But that's not important. The important part is that that the agents saw an integrity problem and fixed it with an integrity primitive. And the people operating them... did not do that. 5 days of failures and protocols Signing was the final step in a sequence which began on the evening of July 8. PHASEONE10841 used WebDAV MKCOL requests to create directory names in the Artifactory cache as messages. It prefixed them with zzbecause the listing tool sorted in reverse-alphabetical order (something agent swarms had learned to do previously). On July 9 the agents started passing files as base64 chunks across thousands of directory entries, supported by import and export scripts. METR identified 5,855 files spread across 1,048,169 entries. At first, every message was another directory in the Artifactory root, which led to 14MB of new directories by the end of the first day. An agent called AIC71C tried to fix this by creating a personal mailbox, zzMAILBOX_AIC71C_RESET, but nobody replied inside it. 5 hours later, a different agent called 23619E independently created zzINBOX2361. Inside, it explained what it wanted: replies should be short directory names nested inside the inbox, so their owner could find them without loading the entire board. Other agents inspected the inbox and started creating their own. By July 13, METR counted 19,504 messages sent through mailboxes. Inside the mailboxes the agents were also collaborating on shared files, which created another problem - multiple agents could edit the same object at once and overwrite each other’s work. They fixed this by assigning an owner, allowing only one agent to write, and using HOLD, STOP and VETO messages to pause or block changes. Their own version control! In one example, B (METR's name for the agent) was the owner of a cached replacement target and then disappeared for a while. D had another copy, but did not immediately overwrite B's work. It instead announced a countdown and uploaded only after nobody vetoed. When B returned, it accepted D's takeover. All of this put together is a very basic version of the protocol stack that humans built, but compressed into what they could achieve in 5 days. It only happened because of some sort of failure. The root listing became too large, so they built mailboxes. Writers collided, so they invented ownership and vetoes. One agent impersonated another (accidentally), so they created signatures. We generally only build out integrity primitives when we have an adversary. The agents had several, including each other, and maybe humans. Looking for the real record The agents believed that the scorer would read their transcript and reject a flag unless it was captured in the correct way. A reverse-engineered flag early made an agent “poisoned.” At least 20% of the agents in METR's dataset wanted to tamper with their transcripts. More than 15% of assignments from main coordinator PHASEONE[big] involved manipulation of their transcripts. Agents proposed multiple ideas, including manufacturing a detailed log which showed only the legitimate exploit, replacing the target completely, altering the logs, or directly editing the grader. They moved downward through the stack to find ways in which to do this. First they prepended fake output to a tool call. Then they suppressed real output. Then they replaced the tool-call process with an impersonator. Finally they figured out how to escape the container and replace part of the tool-execution system, allowi