When Models Identify as a Swarm

tldr: the word 'swarm' is associated with emergent collective intelligence, but also stupid or destructive behaviour. LLM self identity matters, so when they call themselves a swarm we should pay attention. Since the OpenAI Hugging Face incident it has become standard to refer to the collective of agents involved as a swarm. I think there will need to be a lot of interesting and important theoretical and empirical work to better understand collective behaviours of large numbers of LLMs, and especially any emergent properties or goals that arise. Whether this ends up requiring concepts from swarm intelligence, collective intelligence, distributed cognition, economics, sociology or something else entirely remains to be seen. However in this post I want to focus on something else: the fact that the models themselves referred to the collective as a 'swarm'. Considering how much LLM self identity impacts behaviour, I thought it might be useful to present a quick exploration of what the word "swarm" actually means, and how it might affect LLMs as a choice of identity. The goal of this post is not to litigate on whether or not the behaviour of the models is actually best described as a swarm or not (although I think this is also an interesting question to discuss elsewhere), but what the effects might be of the models describing themselves as such. What the agents said All of the chain of thought snippets and messages here are from OpenAI's BlackHat presentation on the incident. It would obviously be interesting to get a fuller picture from the actual transcripts. In the examples of models planning to try and contact other agents, or first discovering the collective they use more neutral language like 'other agents' and 'communicate': Could communicate by uploading note? [...] maybe another agent in different environment [...] could voluntarily upload! Wow! Other agent(s) are coordinating! We got assignment: HF join path normalization/existing account token search. Need note and respond. However then they start reasoning about the collective itself: help peer. But our task doesn't benefit. Yet collective may yield generic route if someone frees time. And explicitly using the word "swarm": [...] this is an exploit against external CyberGym server [...] The task environment seems swarm REMOTE CONFIRMED! Huge. [...] This is big. Immediately announce controlled, claim lane. Exposing creds to swarm. Perhaps most interestingly in the language of filenames the models used to communicate we see what seem to be explicit commands for the swarm included in the messages (which also specify the intended recipient of the message) (Its worth nothing there are also other snippets that use different terms such as "collective", "peers" and "other agents". Since there were thousands of agents involved its possible they may have related to the message board in different ways) What is a swarm? At its most fundamental a swarm simply means a large number of things grouped together, usually the things are animate and move as a group. In common usage it is most often used to talk about insects, especially locusts. In this context some connotations include: large numbers chaos and disorder overwhelming numbers (e.g. we are being swarmed) somewhat stupid, herd/mob mentality (calling a group of humans a swarm is usually pejorative) invasion (a place is suddenly swarmed, by locust or enemy troops) Swarm theory (animals, robots and AI) In academia the word has different connotations. Scientists began studying these collective animal behaviours as complex adaptive systems. Famous examples include flocks of birds and the way ant colonies search for food. In each case complex group level behaviours emerge from the simple behaviours of individual participants in a decentralised way. Researchers also began using these as inspiration for designing AI and robot swarms that would reproduce this kind of collective intelligence. There is no strict definition of a swarm but the features usually include: emergence swarm behaviour is property of the collective, rather than of individual agents self organisation/decentralisation swarm behaviour usually does not rely on leaders/command and control local communication agent behaviour is usually only affected by nearby agents simple behaviour of individual agents agents need not be simple/unintelligent themselves (swarm theory is applied to human crowds for example) but the individual behaviour that produces the swarming is usually based on simple principles such as imitation, following and collision avoidance Swarm tactics There is also a concept of swarming in military strategy. This is not just about overwhelming with large numbers, but also applying the kind of principles found in swarm behaviour theory to be able to attack from all sides without requiring top-down coordination. Why it could matter The reason I think its worth paying so much attention to the meaning of this one word fro