How to talk about extinction

There’s a controversial and unsettled debate in Communication Studies and Psychology about the utility of existential risk framing. This debate has been hashed out for years in the climate change space. Roughly, the two sides are: Pro x-risk framing: (1) It’s true, so rhetors should state the truth; (2) fear is motivating; (3) if our policy goal is to mitigate x-risk, focusing on non-existential risks pushes the public toward policy solutions that don’t solve those existential risks. Anti x-risk framing: (1) It’s viewed by many as hyperbolic and not relevant to their everyday lives; (2) making people fearful doesn’t necessarily translate into action and can be demobilizing; (3) it trades off with a focus on smaller, higher-probability risks. There’s certainly more nuance, but that’s a rough sketch. If you want to take either side of the debate, you can find evidence to support your case. Instead of re-engaging this debate, I want to push the conversation in AI safety in a different direction. This is motivated by two realities: AI safety orgs and advocates are, and are going to continue, using x-risk framing. There’s ample room for new message testing, and limiting our focus to pro- vs. anti-x-risk misses opportunities to improve x-risk framing. I think the two most pressing questions in this domain are: Which x-risk scenario provides the best opportunity for persuasion? We’ve got options: cyber, bio, resource stripping, concentrated power, paperclips, etc. Instead of testing those against mundane risks, test them against each other.[1] When previous studies have found advantages for “everyday” and “immediate” risks, they may have been picking up on concreteness or relatability. Now that we’ve had warning shots, it may be easier to make existential risk more concrete, familiar, explainable, and personally relevant. Messenger testing. Maybe doctors are better at bio-risks, and AI scientists are better at misalignment risks? Create a study where the message is fixed, but the messenger varies. Measure perceived expertise and trust. A more robust study should test whether some risks have better messengers. Are we at a saturation point with “I left an AI firm, and here’s what I fear…”? I don’t have strong leans on the answers to these questions, but they’re worth testing. If you find yourself in an x-risk-centered conversation before someone takes up the mantle on these tests, here are some easy tips informed by evidence in other domains: Explain the x-risk in a concrete manner. Explain the harm so a person can understand how it could threaten their life, or the lives of their family and friends. Laundry-listing every vector for extinction is less important than describing one in a way that sounds plausible. Link existential risk to an action item. People might not feel much hope about AI after your discussion, but give them reasons to feel hope about their ability to take personal action. Be clear about what constitutes a successful message. Are you trying to get someone to support a specific candidate? A specific policy? Sign a petition? Click on a link? Donate money? Just convincing someone to believe in AI risks, and stopping there, is not really the work that needs to be done. I welcome feedback, and appreciate you all reading! [1] We’ve got a start here, but this survey combined cyber/bio and could certainly use an update. Discuss

Full text under the publisher's open licence · 510 words