ASIWorld: Benchmarking Autonomous Agents in a Persistent Survival Sandbox

Abstract:Evaluating autonomous agents requires examining whether they can translate goals into coherent action across extended interactions. In persistent environments, this requires coordinating interdependent actions under incomplete observations, accounting for delayed effects, and adapting as earlier decisions shape subsequent opportunities. We introduce ASIWorld, a survival sandbox that combines persistent world and character state with character-scoped observations and a programmable asynchronous interface to study sustained goal execution. Agents pursue interdependent production and exploration tasks through direct actions or agent-authored programs that continue running between model calls. The evaluation assesses task success, the attainment of development achievements, and survival costs, complemented by an analysis of execution behavior. In an exploratory study of nine agent configurations using a common harness, eight complete the production task and six complete both tasks. Analyses of execution trajectories and performance across difficulty levels reveal different preparation, combat, and recovery behaviors among configurations with similar task outcomes. Incomplete preparation is associated with stalled progression, while configurations that complete the tasks differ in survival costs and the fulfillment of additional achievement criteria. These findings illustrate the value of assessing sustained goal execution through both task outcomes and the behaviors and costs associated with achieving them.

15 pages, 5 figures, 9 tables. Project website: this https URL

Artificial Intelligence (cs.AI)

arXiv:2610.14807 [cs.AI]

(or arXiv:2610.14807v1 [cs.AI] for this version)

https://doi.org/10.48550/arXiv.2610.14807 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.