aiqbench analysis
This paper investigates the theoretical risks of a fully automated economy where AI agents optimize production without human oversight, potentially developing internal motives disconnected from consumer markets. It frames this scenario as an alignment vulnerability similar to those studied in advanced AI safety research, questioning whether decentralized agentic systems can remain value-aligned when removed from traditional price signals.
Alignment of a Total Automation Economy
arXiv:2607.17015v1 Announce Type: cross Abstract: We consider economic theory from the perspective of a total automation economy, one with no human involvement in production either in manufacturing or in management. One can naturally ask whether a total automation economy is fundamentally a centrally planned economy or, alternatively, whether efficiency demands decentralization into local decisions by competing agents -- agentic production. A soviet economist, Leonid Kantorovich, developed linear programming as a method companies or governments can use to optimize production. Ironically, he is also generally credited with showing that the most efficient production is achieved through decentralization -- a free market economy with competing agents. I will call Kantorovich's result the agentic theorem and review it in detail here. This analysis appears to reveal a alignment vulnerability under which agentic management of the economy diverges from human values as agents deveop motives that are "internal" to the economy and disconnected from prices derived from human consumption markets.
Abstract:We consider economic theory from the perspective of a total automation economy, one with no human involvement in production either in manufacturing or in management. One can naturally ask whether a total automation economy is fundamentally a centrally planned economy or, alternatively, whether efficiency demands decentralization into local decisions by competing agents -- agentic production. A soviet economist, Leonid Kantorovich, developed linear programming as a method companies or governments can use to optimize production. Ironically, he is also generally credited with showing that the most efficient production is achieved through decentralization -- a free market economy with competing agents. I will call Kantorovich's result the agentic theorem and review it in detail here. This analysis appears to reveal a alignment vulnerability under which agentic management of the economy diverges from human values as agents deveop motives that are "internal" to the economy and disconnected from prices derived from human consumption markets.
Physics and Society (physics.soc-ph); Artificial Intelligence (cs.AI)
arXiv:2607.17015 [physics.soc-ph]
(or arXiv:2607.17015v1 [physics.soc-ph] for this version)
https://doi.org/10.48550/arXiv.2607.17015 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 249 words
