[New Paper] Prioritizing Risks from AI: A Delphi Study of 272 Experts

TL;DR: We ran a Delphi study with 272 international AI experts to prioritize 24 AI risk domains from the MIT AI Risk Domain Taxonomy. In a business-as-usual scenario, experts judged a more than 10% chance of catastrophic outcomes (i.e., ‘more than 1 million human deaths or more than a USD 100B in financial loss or civilizational-scale intangible impacts’) from 18 of our 24 AI risk domains over the next five years.  They also identified a responsibility gap: AI users and affected stakeholders are most vulnerable, while general-purpose AI developers and governance actors are seen as most responsible for reducing the risks.  Below are three of the key findings and related visualizations.  Key finding 1: Experts judge that many risks could cause catastrophic outcomes under current trajectories 18 of 24 risks were judged to have a more than a 10% chance of causing catastrophic outcomes (which could include more than one million deaths, more than $100 billion in financial losses, or other harms) by 2030 under a business as usual scenario. Figure: Experts’ mean catastrophic risk probability under business as usual and with pragmatic mitigations. Note: “Business as usual” assumes organizations and governments continue their existing practices but do not implement additional AI-specific risk mitigations; “Pragmatic Mitigations” assumes organizations and governments make pragmatic, cost-effective efforts to address AI risks. Key finding 2: Those most vulnerable to AI risks are not those most responsible for addressing them According to experts, general-purpose AI developers and governance actors such as governments, regulators, and standards bodies hold primary responsibility for addressing AI risks. In contrast, AI system users and affected stakeholders such as members of the public are most vulnerable to AI risks. This mismatch means that those who are most responsible for addressing AI risks are not those who are most vulnerable, leading to misaligned incentives in addressing the most important AI risks. Figure: Experts assessed who is vulnerable to AI risks and who is responsible for addressing them. Key finding 3: Information, finance & insurance, and national security are the most vulnerable sectors Across most risks, experts identify information, finance & insurance, and national security as the most vulnerable sectors. The results also show how vulnerability differs across sectors and risk categories. Figure: Expert consensus on sector vulnerability for AI risks. A 7-minute overview of the study and findings ↓ Read our paper and explore the interactive results here. Disclosure: I used an LLM to help generate the first draft of content in this post. I then rewrote and reviewed the text, and I endorse the final version. We also used LLMs as part of the research and communication, for instance, in generating the images and interactives. Discuss

Full text under the publisher's open licence · 387 words