Superintelligence and the Triple Risk of an AI Race

Abstract:On many estimates, artificial general intelligence (AGI) is likely to emerge within the next few decades or even sooner. Once it does, it may quickly surpass human intelligence, possibly by a huge margin. Some believe that the country that develops AGI first could attain global dominance. This prospect risks setting off intense competition between the United States and China. Indeed, a narrative has already developed that the superpowers are locked in an AI race, with the Trump administration proclaiming its determination to 'dismantle unnecessary regulatory barriers' to winning it. We argue that this approach presents three profound dangers. The first two are familiar from past arms races: being overtaken by a rival or triggering a war. The third is new: losing control of human beings' future to agents far more intelligent than we are, which could endanger our very survival. Striving to get to AGI first risks a race to the bottom on safety practices and regulation concerning them. China does not yet appear bent on racing to superintelligence. The United States should prioritise both unilateral and co-operative efforts to improve AI safety and avoid provoking a race that all humanity could lose.

Computers and Society (cs.CY); Artificial Intelligence (cs.AI)

arXiv:2610.17085 [cs.CY]

(or arXiv:2610.17085v1 [cs.CY] for this version)

https://doi.org/10.48550/arXiv.2610.17085 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.