Warning Shots for AI Existential Risk: Will society answer the wake-up calls it receives?

Introduction If anyone builds superintelligence, everyone dies. That is, it is extremely reckless to build AI systems more intelligent than all humans, given anything like our current technical understanding. To address this, MIRI and others have suggested an immediate and decisive halt to pushing the frontier of general AI capabilities. We think the existing arguments are strong, but many others are waiting for more evidence. This is explainable in part by the large contrast between the harms caused by current and future AI systems. People in AI governance often talk about warning shots: events which could solve this conundrum, often by causing less-than-existential harm. This post attempts to analyze the likelihood of a successful warning shot for superintelligence, where success requires that the warning shot lead to effective action. We first propose a framework for predicting whether a warning shot will occur and be successful. We apply the framework to analyze commonly-discussed warning shots: AI-caused pandemics, various military applications, and cyber misuse affecting critical infrastructure. In considering these cases and the framework, we conclude that it is quite plausible that there will simply be no warning shot which generates an appropriate response. Therefore, we think the AI governance community and policymakers should not be waiting for warning shots; the time for action is now. A Framework for Predicting Whether Warning Shots Will Work To motivate an effective response, a warning shot will need to meet these criteria1: Likely: To motivate an effective response, a warning shot will need to occur. From our perspective, we ask whether the event is plausible given our current understanding. Timely: It occurs early enough to be reacted to. Conditional on it occurring, will it precede the closing of the window for an effective preventive response? Visceral: The event makes an impression. It is noticed. Much of the discourse on warning shots focuses on the power of visceral harms to motivate a response, but this is not sufficient on its own, hence the rest of the framework. Sudden: The event happens or the visceral impacts occur over a short timeframe, so that individuals and societies experience “a moment”. The alternative dynamic is referred to as “frog boiling”, where individuals and society fail to notice the event or its implications because they gradually get accustomed to a new normal. Society might wake up without an event being sudden, but it seems much harder. Unexpected: To change an observer’s beliefs, an observation needs to not have been totally predicted by that observer. That is, our audience has to notice that something happened which they didn’t expect to prompt reexamination of their current beliefs. It will likely need to be substantially outside of what they previously considered plausible to not be written off as a (un)lucky or one-off occurrence. Attribution: It is perceived as being caused by AI; difficult to deny that AI progress is materially involved. It is not critical that AI was actually responsible. Backlash: It motivates hostility toward continued AI development rather than enthusiasm. This is required in order to motivate restraints on AI development which actually make a difference in averting existential risk. Backlash can be undermined when the warning shot indicates that further investment in, or deployment of AI is warranted. This might happen when response is channeled into societal resilience rather than prevention. Response Availability: Is a well-formed, legitimate policy response already available with well positioned advocates? Have policies of first resort been tried and shown to be ineffective? Crises can open windows for new policy, but these are often filled with policies which are ready and perceived to be easy. This is one area where the AI governance field can have a large impact, and where there is much work that has already been done. We exclude this from our analysis below, as it is largely agnostic to the properties of individual warning shots, but note it here, as in practice a warning shot being converted into policy may depend on the availability of a response. For each class of warning shot below, we give a coarse grade of pass, uncertain, or fail to each criterion. Which warning shots are expected to be strong? AI-Caused Pandemics COVID-19 caused between 15 million and 35 million extra deaths worldwide. It is also an example of tremendous harm not leading to an appropriate preventive response: society could purchase effective resilience against a future pandemic through any number of methods (stockpiling PPE, vaccine distribution pre-logistics, wastewater monitoring, hardening buildings against airborne spread through UV treatment and/or filtering), but has done surprisingly little of this2. How likely is AI to facilitate a human-caused pandemic? Cutting edge AI systems have already demonstrated the ability to expand the number of a