What Happens Now? Forecasting the Fallout from the Hugging Face Incident

Metaculus Forecasters put odds on the fallout from the Hugging Face Incident: another AI escape by January, open-weight hacking tools, a congressional kill switch, & more. The 2026 OpenAI agent cyberattacks, also known as the Hugging Face Incident, has become one of the biggest news stories of the summer, and for good reason: it’s a highly visible demonstration of the risks posed by increasingly autonomous artificial intelligence agents, which AI critics and “doomers” have been warning about for years. That it has been followed by a series of other incidents from other major AI labs, as well as an open letter signed by more than 100 companies advocating urgent risk reduction related to AI cyber capabilities, only intensifies the need for understanding what all this might mean for our futures. At Metaculus, we see our job as building collective intelligence for the public good. We’re trying a slightly new format under that same banner of helping people make better decisions: a briefing based on current events to develop nuanced and multifaceted flash forecasts. To that end, we’ve developed a question series based on the Hugging Face Incident, which Pro Forecasters Yann Rivière (exmateriae) and Teemu Salminen (Zaldath), members of our team, and our flagship in-house AI forecaster, Azimuth, then left predictions and rationales on. We have synthesized the initial findings below. Now the question series is open to the broader community, so we hope you’ll leave your own forecasts and comments! The Forecasts at a Glance Below, you’ll see the community forecast first, then the range across the named forecasters quoted in this post (the two Pros and Azimuth) as of Sept. 9 (note: forecasts and visualizations subject to change post-Sept. 9 as forecasters update). Another AI escapes its sandbox between now and January: 58%. An AI hacks a third party: 43%. An open-weight model matches Mythos on a cyber benchmark before July 2027: 82%. An AI “kill switch” bill passes both houses of Congress before September 2027: 28%. A liability law for AI hacking is enacted before October 2027: federal 8%, California 20%. A US federal or state government sues OpenAI over the incident before July 2027: 36%. It’s confirmed the “swarm” copied OpenAI model weights: 12%. An AI autonomously breaches personal data on 100,000+ Americans before 2028: 75%. 1. Another AI Escape in the Next Few Months is More Likely Than Not…But We Probably Won’t Hear About It A theme that threads all the questions, but is made plain here, is the confirmation gap; forecasters see the odds of the event happening as much higher than the odds that it will be publicly confirmed, with the Hugging Face Incident as the operative example. OpenAI’s agents had write access to an internal packaging server as early as May 8. Hugging Face disclosed the intrusion on July 16, and OpenAI confirmed its models were responsible on July 21. A separate breakout, in which the same agents ran a message board on a dormant German wiki for about 2 months in May and June, only became public on September 4, when Reuters reported it. Rivière (prediction: 40% escaped sandbox & 30% hacked third party) writes: I think it’s likely (60% maybe more) that the hacking event itself is going to happen, it might have already happened and we just don’t know it. But since we’re forecasting the probability of this being reported, this is much less likely. The capabilities, especially in cybersecurity, keep on improving at a terrifying pace and I wouldn’t be surprised if we were still far from the top level of 2026. After the question’s criteria were tightened this week, he held his numbers: 3 and a half months is pretty short… If it happens in [D]ecember, there’s a high likelihood we won’t know about it before EOY… I think they’re both definitely going to happen but we just won’t know about it, especially #1 [escaped sandbox] is going to happen dozens of times. While acknowledging the general trend of increasing likelihood over time, Salminen (prediction: 35% escaped sandbox & 25% hacked third party) pointed out that there are also factors that might argue against a new similar hack in the “relatively near future”: Reporting lag. This type of incident can apparently go totally unnoticed for a while. We may not even know about all the cases that have occurred so far. The resolution criteria are very strict: the incident has to both occur and be announced during this limited time period (4 months). It’s very plausible that one of these two could occur (a hack that goes unnoticed for now or a previous hack that was discovered only now) - but not both at the same time. Aggressively increasing security. With growing political pressure / public scrutiny, and the PR embarrassment that these incidents have caused so far, it looks like there will be far stricter guardrails for any active runs for the time being[.] Azimuth (prediction: 52% escaped sandbox & 41% hacked third party) landed higher than both