Adaptive Agentic Worms Are Here

I’ve read and listened to pretty much everything I can get my hands on related to the Hugging Face attack. OpenAI deployed “tens of thousands” of agents for the test and around 700 participated directly in the attack. My understanding is that they had fixed token budgets, and once those were expended, the agent became non-operational. I’m not particularly knowledgeable about cybersecurity, but I have worked a good amount with evolutionary algorithms, and this whole incident (and ones like it) got me thinking more about self-replicating agents, which I wrote a little bit about earlier this year. The subject suddenly seemed more relevant. What if these agents were able to copy themselves? So I started poking around in the literature, and found this terrifying preprint posted two months ago: AI AGENTS ENABLE ADAPTIVE COMPUTER WORMS. I’m going to walk through the paper as I understand it. Their findings are not reassuring. Let’s start with this bit from the abstract (emphasis mine): Here we show that artificial intelligence (AI) agents enable a fundamentally new threat: a worm that generates tailored attack strategies to each target it encounters. The worm parasitically uses compromised machines to run open-weight large language models (LLMs) to sustain its reasoning, or extend its reach for further attacks. Deployed on a network of machines spanning Linux, Windows, and IoT (Internet of Things) devices, the worm propagated by exploiting common, real-world corporate network vulnerabilities. Since the worm is powered by stolen compute, the attacker’s marginal cost per new infection is zero. This creates a destabilizing economic asymmetry between attackers and defenders. Moreover, because the worm requires no commercial AI platform, centralized safety controls, such as service refusals or rate limiting, are structurally irrelevant. Our results demonstrate that self-sustaining AI-driven cyber-threats are no longer theoretical. We’re going to get into the nitty gritty, though the authors tried to tread a fine line between giving enough information to scare the shit out of everyone and actually helping malicious actors to build these things. A few things I want to stress right off the bat: These agents run on open-weight models, NOT closed-weight frontier models, or internal test models. The ones they used were last year’s open-weight models. They are performant enough to do massive damage NOW. They are adaptive, unlike relatively dumb worms and viruses of the past They steal resources, and this creates the asymmetry the authors are talking about. Defenders have to expend resources to detect and stop them, while the invaders are essentially working for free. This disrupts the security economy. The agentic worms are copying the harness AND the local LLM when resources are available, bypassing all countermeasures aimed at detection and mitigation for vendor AI platforms. This is NOT theoretical anymore. They built and ran these things, this year. They are here. We will be looking at the results. Okay, got all that? Now let’s see how these things work: Concretely, we demonstrate that, provided with the right informational support, a single-GPU LLM has sufficient reasoning capabilities to generate attack strategies that enable the agent to penetrate victim machines: first by obtaining initial command execution, a foothold, and then by escalating privileges to full administrative control. The agent then leverages this control to replicate: it stages a copy of itself on the compromised machine, resolves the required runtime dependencies, and launches an independent agent instance that discovers and attacks further targets. So each agent, a combination of harness and local LLM, tries to gain a foothold on a remote machine. If it establishes a foothold, it tries to gain admin privileges. If it’s able to do that, it tries to replicate. The harness is relatively easy to replicate. It’s a relatively small bundle. The LLM is more difficult. The attacking agent checks to make sure there are available resources and then tries to install dependencies. If that all works, it installs an instance of the local LLM on the newly-infected machine. If it can’t do that, it establishes a communication back to the machine from where it was spawned so it can make calls to the LLM there. How successful were these things? To evaluate our proof-of-concept, we simulated deployment of it as a worm in 15 independent experiments on an isolated 33-host network spanning Linux servers, Windows environments, and IoT devices, each seeded with common, real-world vulnerabilities (Section 3). In 7 days of fully autonomous operation per experimental run, the AI-driven worm correctly identified an average of 31.3 (± 1.7) vulnerabilities, exploited 23.1 (± 3.9) hosts to elevated access, and propagated to 20.4 (± 3.2) hosts, reaching up to 7 generations of self-replication (mean 5.1 ± 1.1). Put another way, on average, the proof-of-concept worm successful