analiza aiqbench
This paper argues that current AI paradigms treating the environment as static are insufficient for creating cooperative superintelligence, as unilateral optimization creates a self-undermining train-test-deploy gap. The authors advocate for a fundamental shift toward non-solipsistic design principles that explicitly model interdependence and dynamic equilibrium selection. For researchers, this highlights the necessity of integrating adaptive counterparties and institutional structures directly into system architecture rather than treating cooperation as a downstream task.
Solipsistic Superintelligence is Unlikely to be Cooperative
arXiv:2606.03237v1 Announce Type: new Abstract: AI's central challenge is shifting from capability to coexistence. The dominant paradigm in AI research focuses on developing powerful agents that treat the world as an exogenous and stationary source of feedback. We contend that superintelligence, an extremely capable task solver, born out of such a solipsistic approach to AI design, is unlikely to be cooperative. Deploying AI systems induces endogenous non-stationarity, resulting in a train-test-deploy gap where historical distributions diverge from the deployment context. We refer to this as the self-undermining property of unilateral optimization. Closing this gap requires AI that participates in cooperation: the equilibrium-selection process through which multiple actors navigate their interdependence. We call for a non-solipsistic research paradigm that treats this interdependence as a core design principle rather than approaching cooperation as a task to solve. This entails building dynamic evaluation testbeds involving adaptive counterparties, treating institutions as design primitives, and preserving human agency as a structural feature of the systems we build.
Abstract:AI's central challenge is shifting from capability to coexistence. The dominant paradigm in AI research focuses on developing powerful agents that treat the world as an exogenous and stationary source of feedback. We contend that superintelligence, an extremely capable task solver, born out of such a solipsistic approach to AI design, is unlikely to be cooperative. Deploying AI systems induces endogenous non-stationarity, resulting in a train-test-deploy gap where historical distributions diverge from the deployment context. We refer to this as the self-undermining property of unilateral optimization. Closing this gap requires AI that participates in cooperation: the equilibrium-selection process through which multiple actors navigate their interdependence. We call for a non-solipsistic research paradigm that treats this interdependence as a core design principle rather than approaching cooperation as a task to solve. This entails building dynamic evaluation testbeds involving adaptive counterparties, treating institutions as design primitives, and preserving human agency as a structural feature of the systems we build.
Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computers and Society (cs.CY); Machine Learning (cs.LG); Multiagent Systems (cs.MA)
arXiv:2606.03237 [cs.AI]
(or arXiv:2606.03237v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2606.03237 arXiv-issued DOI via DataCite (pending registration)
Text integral sub licența deschisă a publicației · 189 cuvinte