aiqbench analysis
This paper introduces SA-WAM, a novel approach that integrates 3D depth information into World Action Models without fine-tuning the underlying frozen diffusion backbone. By effectively encoding unbounded depth signals, it achieves state-of-the-art performance on robotic benchmarks and demonstrates improved real-world control using a UR5 arm. The work advances robot policy learning by proving that geometric priors can be leveraged alongside visual data within a single unified framework.
Spatially Aware World Action Model via Geometric Latent Diffusion
arXiv:2609.02531v1 Announce Type: cross Abstract: World Action Models (WAMs) leverage the capabilities of large-scale pretrained video diffusion models to jointly predict future observations and actions, inheriting rich visual and physical priors from internet-scale video. This has made them a promising paradigm for robot policy learning, yet the prevailing models operate exclusively on RGB observations and do not leverage 3D information. To bridge this gap, we introduce a Spatially Aware World Action Model (SA-WAM), which repurposes a pretrained video model for joint action, RGB, and depth prediction, enabling 3D-aware world modeling and action prediction within a single diffusion backbone. We use a nonlinear encoding that maps the unbounded depth signal into the bounded input domain expected by the frozen VAE tokenizer. This allows us to reuse the tokenizer without 3D-specific fine-tuning, incorporating geometric information without sacrificing the pretrained priors. SA-WAM achieves state-of-the-art results on the RoboCasa and LIBERO-Plus benchmarks, while simultaneously improving future-state predictions. Furthermore, SA-WAM outperforms strong baselines in real-world evaluation using a UR5 robotic arm, with strong gains in randomized environments. We analyze the correlation between world model prediction quality and rollout success, providing insights into WAM performance and avenues for its improvement.
Abstract:World Action Models (WAMs) leverage the capabilities of large-scale pretrained video diffusion models to jointly predict future observations and actions, inheriting rich visual and physical priors from internet-scale video. This has made them a promising paradigm for robot policy learning, yet the prevailing models operate exclusively on RGB observations and do not leverage 3D information. To bridge this gap, we introduce a Spatially Aware World Action Model (SA-WAM), which repurposes a pretrained video model for joint action, RGB, and depth prediction, enabling 3D-aware world modeling and action prediction within a single diffusion backbone. We use a nonlinear encoding that maps the unbounded depth signal into the bounded input domain expected by the frozen VAE tokenizer. This allows us to reuse the tokenizer without 3D-specific fine-tuning, incorporating geometric information without sacrificing the pretrained priors. SA-WAM achieves state-of-the-art results on the RoboCasa and LIBERO-Plus benchmarks, while simultaneously improving future-state predictions. Furthermore, SA-WAM outperforms strong baselines in real-world evaluation using a UR5 robotic arm, with strong gains in randomized environments. We analyze the correlation between world model prediction quality and rollout success, providing insights into WAM performance and avenues for its improvement.
Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
arXiv:2609.02531 [cs.CV]
(or arXiv:2609.02531v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2609.02531 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 288 words
Related coverage
SG-WAM: Self-Guided World Modeling in Geometry-Aware Policy Space
arXiv · Aug 4, 2026
AffordDrive3D: Affordance-Aware World-Action Modeling with Spatial Understanding
arXiv · Oct 10, 2026
GeoWorldAD: Geometry World Action Model for Autonomous Driving
arXiv · Jul 21, 2026
GeoSem-WAM: Geometry- and Semantic-Aware World Action Models
arXiv · Jun 3, 2026