aiqbench analysis
This paper introduces ACG-WAM, a novel world action model that jointly learns visual prediction and robot actions by explicitly targeting geometric consequences of demonstrated sequences. By utilizing an Action-Conditioned Geometric Joint-Embedding Predictive Architecture (ACG-JEPA), the method improves policy learning on both simulated and real-world robotic tasks while avoiding data leakage from future observations. The approach demonstrates superior performance over existing baselines in clean scenes and partial completion scenarios, offering a robust framework for sim-to-real transfer in robotics.
ACG-WAM: World-Action Modeling via Action-Conditioned Geometric Latent Prediction
arXiv:2610.06965v1 Announce Type: new Abstract: World action models jointly learn visual predictionand robot actions, providing a way to use observations ofscene evolution for policy learning. Their video and actionlosses, however, provide no explicit target for the geometricconsequences of a demonstrated action sequence. Moreover,visual features taken after temporal attention can contain futureobservations, making them unsuitable as the sole current visualinput to an auxiliary predictor. We introduce ACG-WAMand its auxiliary objective, the Action-Conditioned GeometricJoint-Embedding Predictive Architecture (ACG-JEPA), whichpredicts geometric features at several horizons from the currentobservation and intervening actions, using the future slot of afrozen VGGT encoding of each current and future image pairas the target. We apply this supervision from the head and wristcameras to a shared visual embedding before temporal mixing,and remove the teacher and auxiliary modules at inference.On 50 RoboTwin 2.0 tasks, ACG-WAM achieves 93.46%success in clean scenes, with the best randomized success(92.68%) and mean across both settings (93.07%) among thecompared methods; across three tasks on a real robot, itachieves 85.00% success and 91.67% partial completion score,exceeding Motus by 10.00 and 9.17 percentage points, respec-tively. Code:https://github.com/RoboOpus/ACG-WAM.Website:https://RoboOpus.github.io/ACG-WAM.
Abstract:World action models jointly learn visual predictionand robot actions, providing a way to use observations ofscene evolution for policy learning. Their video and actionlosses, however, provide no explicit target for the geometricconsequences of a demonstrated action sequence. Moreover,visual features taken after temporal attention can contain futureobservations, making them unsuitable as the sole current visualinput to an auxiliary predictor. We introduce ACG-WAMand its auxiliary objective, the Action-Conditioned GeometricJoint-Embedding Predictive Architecture (ACG-JEPA), whichpredicts geometric features at several horizons from the currentobservation and intervening actions, using the future slot of afrozen VGGT encoding of each current and future image pairas the target. We apply this supervision from the head and wristcameras to a shared visual embedding before temporal mixing,and remove the teacher and auxiliary modules at this http URL 50 RoboTwin 2.0 tasks, ACG-WAM achieves 93.46%success in clean scenes, with the best randomized success(92.68%) and mean across both settings (93.07%) among thecompared methods; across three tasks on a real robot, itachieves 85.00% success and 91.67% partial completion score,exceeding Motus by 10.00 and 9.17 percentage points, respec-tively. Code:this https URL.
Robotics (cs.RO)
arXiv:2610.06965 [cs.RO]
(or arXiv:2610.06965v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2610.06965 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 269 words
Related coverage
Geometric Action Model for Robot Policy Learning
arXiv · Jun 16, 2026
SG-WAM: Self-Guided World Modeling in Geometry-Aware Policy Space
arXiv · Aug 4, 2026
J-LAW: Joint Localization and Action-Conditioned World Modeling via Coupled Latent Factor Graphs
arXiv · Sep 2, 2026
Does Latent Planning Survive Point Clouds? Action-Conditioned JEPA World Models for Geometric Observations
arXiv · Sep 1, 2026