analiza aiqbench
The article examines the theoretical significance of instrumental convergence within AI safety frameworks, specifically regarding resource-seeking behaviors and catastrophic outcomes like pandemics. It seeks community consensus on the prevalence of these risks as a central theme in current safety research rather than discussing specific products or policy events.
How important is instrumental convergence?

Published on October 9, 2026 7:49 AM GMT Instrumental convergence is the thesis that sufficiently intelligent agents with a high proportion of goals will tend to pursue certain similar instrumental goals, such as resource- and power-seeking, self-preservation, and recursive self-improvement. I'm writing on the subject, and want to get a sense of how important the community thinks this is in the context of AI safety. So this needs two polls for comparison. Defining 'catastrophe' as 'a >50% drop in human population in under a decade' and 'AI-caused' as 'some event clearly tracing to the action of one or more AIs alone or of humans using AI' (so e.g. an artificially created pandemic would count). (I couldn't see a way to get a legend on the poll, but there are 20 values available, so they're implicitly increments of 5%) Please give any context that you think matters. Ty! Discuss
Citește articolul complet pe Effective Altruism ForumSubiecte conexe

Social Theory and Agent Foundations
LessWrong · 8 oct. 2026
Lock-In Risk Needs More Researchers; Here's Where to Start
LessWrong · 17 iun. 2026
In open RLVR, “improvement” depends on the instrument — a small GRPO testbed separating what training optimizes, measures, and teaches
LessWrong · 15 iun. 2026
The long arc of alignment: second-order instrumental convergence
LessWrong · 11 iun. 2026