Brain preservation as existential risk reduction

“…it will actually be possible to cure most human disease in ~5-10 years” -Dario Amodei, CEO of Anthropic, writing on X, August 2026. About 170,000 people will die today, the overwhelming majority of them from illness. Amodei is claiming that on the back of AI progress, most of that could stop. It’s a bold claim, but with every passing year it sounds slightly less outlandish than it did the year before. In the past six years, large language models have gone from producing plausible-sounding paragraphs to discovering new mathematical theorems, finding zero-day security vulnerabilities, and designing new viruses. They remain strange and patchy - superhuman at some tasks while hopeless at others - but the direction has been consistent, and if it holds, they will eventually be better than us at everything. While “past performance is no guarantee of future results”, as the saying goes, previous predictions that AI progress would stall have not fared well. Should they succeed in creating machine supergeniuses, the CEOs of the companies building them have been explicit about how they want to use them. In Machines of Loving Grace, Amodei writes of people living after AGI: “I hope that any mention of disease will sound to them the way scurvy, smallpox, or bubonic plague sounds to us. That generation will also benefit from increased biological freedom and self-expression, and with luck may also be able to live as long as they want.” Similarly, Demis Hassabis of Google DeepMind, last year on 60 Minutes: “I think one day, maybe we can cure all disease with the help of AI... Maybe within the next decade or so. I don’t see why not.” This isn’t only hype. Hassabis has a Nobel Prize for developing AI that can reliably predict the structure of proteins, compressing a process that used to consume an entire PhD per structure into about a minute of compute apiece. If technology advances to the point where AI can design its own experiments, run them, analyze the resulting data, and choose the next experiment to proceed with, at each step performing better than the best human biomedical scientists, there is no good way to bound how fast biomedical science could move. Assuming this scenario comes to pass (a big assumption!), the timing of this potential acceleration matters enormously - nobody who dies today benefits from a cure that arrives next year. If pulling that timeline forward by twelve months would mean over 60 million additional people are still alive to enjoy the results, then there’s a serious case for building these AIs as fast as we possibly can. As tech billionaire and self-described accelerationist Marc Andreessen puts it in The Techno-Optimist Manifesto: “We believe Artificial Intelligence can save lives – if we let it. Medicine, among many other fields, is in the stone age compared to what we can achieve with joined human and machine intelligence working on new cures. There are scores of common causes of death that can be fixed with AI, from car crashes to pandemics to wartime friendly fire. We believe any deceleration of AI will cost lives. Deaths that were preventable by the AI that was prevented from existing is a form of murder.” But as stories of genies and golems warn us, it is hard to ensure a creature more powerful than you actually does what you desire. If we build machines cleverer than ourselves, how do we make sure they end up wanting what we want them to want? Natural selection is the standard illustration of why this is hard. Selection pushes organisms to leave as many fertile offspring as possible, but has no way of writing that abstract goal into a brain directly. What selection does instead is keep whichever brains happen to leave more offspring, and what those brains typically end up containing is a proxy - a desire for sex. In the case of humans, that objective worked as a stand-in right up until we invented contraception. Knowing exactly what evolution built you for changes nothing either: you can understand the whole story and still take the pill, because caring about your descendant count was never what got wired in. This is the alignment problem: the difficulty of crafting a system whose goals are the ones you intended, when goals can only ever be specified indirectly by selecting on some measurable stand-in for what you actually want. You might think this is a problem specific to blind processes like evolution, rather than for machines with human designers. That might hold if we created modern AI the way we historically wrote software, as explicit rules with an author behind each one. But modern AI is grown rather than written. Engineers pick an architecture, then choose some way of scoring the system’s output - did it get the right answer, did the user approve of it, did the code run - and then a training process nudges several hundred billion numbers up or down, over and over, keeping whatever makes the score go up. None of those numbers are legible to the engineers who grew th