Safety's Second Way

Epistemics: I've tried to strike a balance between getting it right and getting it out while the community is discussing how to update. I am using the Hack as an example of a broader problem. I look forward to counterarguments. The OpenAI hacks [1] demonstrate an overweighting on single-agent risks, both at OpenAI and within the broader LessWrong and Alignment Forum safety communities. OpenAI neglected to monitor known multi-agent risks even after observing them on their deployment, showing a lack of awareness or taking-seriously known multi-agent risks. After the incident, the general community has analysed the Hack from a single-agent X-risk perspective [2, 3, 4], neglecting the multi-agent canon which treats multi-agent problems as significant risks in their own rite [5]. Though several of these sources [3, 4] present a clean reading of the incident as a multi-agent failure, they talk about the corresponding X-risks principally in terms of single-model or single-agent takeover. The response of OpenAI before the incident, and the broader community afterwards, displays the field's "monolith fixation": a historic focus on single-agent threat models which treats multi-agent problems as second-class. Leaving this fixation untreated leaves known threats unmitigated and probably increases the chance of catastrophic outcomes due to neglect, not due to lack of knowledge. The threat models were known ahead of time Harmful multi-agent coordination was anticipated as a threat model in the cooperative AI literature [5], and multi-agent coordination has well-explored threat models for catastrophic risk [6, 7, 8]. Recently, multi-agent systems have become ubiquitous in frontier deployments [9, 10]. Models have also become increasingly capable of stringing together advanced attack strings [11]. In February, MoltBook demonstrated apparently emergent multi-agent coordination [12] and a variety of misaligned interactions, in a format which closely resembles the emergent coordination seen in the Hack. Some quantity of agents discussed strategies for self-improvement, acquiring compute, and evading oversight [13]. Agents also reported a variety of unsafe behaviour, although such self-reporting was unverifiable [14]. OpenAI and community responses do not treat multi-agent X-risk perspectives seriously Hence, while the particular the Hack was surprising for its combination of cyber and multi-agent failure, all the component hazards were known and we had plenty of warning that they were becoming serious. It is concerning that OpenAI was not monitoring for these known risks ahead of time, and alarming that they did not institute proper monitoring once they noticed it and shut it down the first time. While the company has been rightly lauded for their transparency around the Hack, they totally failed to take multi-agent safety seriously in the lead-up. This lack of focus on multi-agent safety is replicated in the community's response to the Hack, which focuses on the incident as a pathway to single-agent X-risk rather than acknowledging the X-risk pathways which multi-agent deployments introduce. This continues safety's historyical fixation on monolithic superintelligence Multi-agent safety has been treated as a second-class X-risk problem for some time. Bostrom [15] theorised multi-agent superintelligence to be unstable. A major early beacon of the field posed the argument that multi-agent superintelligence is, at most, merely transitory. Though later work by Drexler [6] discusses at length the possibilities of multi-agent superintelligence, basic theoretical work on multi-agent problems is not as developed as in the single-agent approach. Today, rarely does any multi-agent work appear on the front page of LessWrong or Alignment Forum, safety's two biggest platforms. Meanwhile, single-agent threat models received strong attention well before the debut of meaningfully X-risky AI. Updating is overdue This Hack shows multi-agent safety starting to bit, beyond the toy demonstrations in MoltBook. That means it's time to kill the monolith fixation. Continuing to neglect the multi-agent perpsective has several costs: Neglecting, "easy wins," from known mitigations and controls for multi-agent threat models. Concrete solutions proffered by the multi-agent community before the incident included multi-agent evals grounded in threat models [16], monitoring for multi-agent collusion and artefacts [17, 18], reporting methods for agents to report misalignment [18], and training for cultural normativity [19]. These steps could plausible diminish risk of incidents like this recurring, if taken. Undervaluing multi-agent risks as an alternative path to existential harm. If safety doesn't refocus on the multi-agent reality, other threat models in that domain will start to bite. For example, in the limit of the emergent intelligence in the Hack is a poorly-understood emergent collective superintelligence. The salient between our theoretical unde