Machine Organizations

OpenAI is nothing without its people On November 20th, 2023, this was tweeted by many OpenAI employees as a sign of solidarity with Sam Altman in his conflict with the then-board. OpenAI published a blog post yesterday about Research acceleration; they have successfully hit the target of an ‘automated research intern’ that they set for themselves and hope to have an automated AI researcher by March of 2028. At some point in the foreseeable future, OpenAI could be something without its people. But what? My default medium-term scenario for continued AI escalation is still global takeover where humans are entirely displaced, but it seems worth investigating scenarios wherein AIs and humans coexist, at least briefly.[1] Historically I have thought this case was not particularly relevant. It seemed like an AGI that became significantly economically competitive would also be significantly strategically competitive, because of underlying general capacities, and the transition period thus relatively short. But this is perhaps not taking into account Moravec’s paradox. Humans have long used machines to accomplish their ends. Many tasks currently performed by machines were once performed by humans, and a large fraction of our modern abundance comes from the ability of machines to perform tasks more efficiently, quickly, and reliably than humans. The classical examples of this are humans employing machines as tools to replace humans that they were employing as tools. The elevator operator who moves the elevator to the floor you ask for can be easily swapped out for a simple circuit that performs the same function. But the ‘nervous system’ of a system or organization is also built out of tasks. At a vintage taxi company, a customer calls the dispatcher, who then decides which driver to send to them. But at Uber, software replaces the dispatcher, and so the human employee is taking orders from a machine boss. At Tesla’s Robotaxi, both the dispatcher and the driver have been replaced by machines (while other support tasks are presumably still accomplished by humans). In military contexts, much has been written about the rise of drones in warfare, replacing humans at the tip of the spear. But modern militaries are logistical and informational organizations; battlefield command & control and intelligence gathering is where current artificial intelligence shines. The picture of human generals ordering around robotic soldiers seems less likely than one where computer generals order around a mixture of human soldiers, dumb machines, and smart machines. One might object that decision-makers will be unlikely to replace themselves with machines. A fair point! But in our current system, every decision-maker is put in place by other decision-makers, who may field them uncompetitive with machine alternatives. What prevents machines from being superhuman at legitimacy?[2] What prevents machines from superhuman performance at managing investments, or managing companies? Marc Andreessen famously predicted that venture capital would be one of the last jobs to be automated in 2025, because of the importance of intangible factors; it is really not clear to me that those intangible factors are ones where it is impossible to be superhuman.[3] So a machine organization is one where the ‘relevant’ human employees have been replaced by machines. A version of OpenAI which has replaced sama with Sambot, its researchers and sales staff with Galaxy,[4] but still has datacenter technicians, is still fundamentally a machine organization. Some immediate observations: First, from the perspective of the rest of the world, not much has to change in the short term. If OpenAI fires its researchers because models are more competitive, the datacenter technicians will still get paid if they show up and not paid if they don’t. Customers will receive the same level of service, and presumably choose to keep using their services. (It’s not like people are using OpenAI to fund the lifestyles of the OpenAI employees; they’re using them because they get more value than it costs.) Investors will still expect to receive their dividends (if ever issued) or be able to sell their shares to other investors. Over three quarters of the researcher time spent at OpenAI is machine labor instead of human labor;[5] did the outside world notice the shift from when half the labor time was human (back in the long-ago month of June)? Second, it’s no longer easy for the government to interface with OpenAI. If the organization commits crimes, who is the responsible corporate officer who can be put in jail? There might be someone on paper--but if they don't perform the functional role in the organization that corresponds to their title, imprisoning them doesn't actually affect the organization's function. If the government decides to criminalize research that races towards superintelligence, does it have tools to stop the machines? It's currently a legal gray area as to whether or n