analiza aiqbench
LeWAM advances robotics by utilizing Joint-embedding predictive architectures (JEPAs) to create compact latent world models that bypass pixel reconstruction, enabling direct policy execution and significantly faster planning. This approach achieves a substantial improvement in success rates for contact-rich and long-horizon tasks while maintaining competitive performance on behavior cloning benchmarks. The method's ability to learn control-relevant representations makes it a notable step forward for sim-to-real transfer in autonomous systems.
LeWAM: End-to-End World and Action Modeling with JEPAs
arXiv:2610.12818v1 Announce Type: cross Abstract: Joint-embedding predictive architectures (JEPAs) learn compact latent world models without pixel reconstruction. Existing JEPA pipelines typically train visual representations and action-conditioned dynamics before adding a control policy or solver. We introduce LeWAM, an end-to-end JEPA for control with: (1) a joint training objective over next latent state prediction and action flow matching, and (2) an isotropic Gaussian constraint on the latent state, enabling a single model to support direct policy execution and efficient planning by evaluating sampled actions with its learned latent dynamics. Compared with prior JEPAs in visual planning, LeWAM improves success rate from 28.6% to 89.7% on contact-rich tasks and from 10.9% to 46.2% on long-horizon tasks, and plans 32.3 times faster. On behavior cloning tasks in simulation and physical robots, LeWAM is also competitive with established methods. Interpretability analyses show that LeWAM learns control-relevant representations and action distributions effectively guiding test-time planning and control.
Abstract:Joint-embedding predictive architectures (JEPAs) learn compact latent world models without pixel reconstruction. Existing JEPA pipelines typically train visual representations and action-conditioned dynamics before adding a control policy or solver. We introduce LeWAM, an end-to-end JEPA for control with: (1) a joint training objective over next latent state prediction and action flow matching, and (2) an isotropic Gaussian constraint on the latent state, enabling a single model to support direct policy execution and efficient planning by evaluating sampled actions with its learned latent dynamics. Compared with prior JEPAs in visual planning, LeWAM improves success rate from 28.6% to 89.7% on contact-rich tasks and from 10.9% to 46.2% on long-horizon tasks, and plans 32.3 times faster. On behavior cloning tasks in simulation and physical robots, LeWAM is also competitive with established methods. Interpretability analyses show that LeWAM learns control-relevant representations and action distributions effectively guiding test-time planning and control.
Robotics (cs.RO); Artificial Intelligence (cs.AI)
arXiv:2610.12818 [cs.RO]
(or arXiv:2610.12818v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2610.12818 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 243 cuvinte
Subiecte conexe
LeWAM: A JEPA World Action Model with Diffusion-Steering-Based MPC
arXiv · 10 oct. 2026
Latent evolving World Action Model
arXiv · 25 sept. 2026
World Models and Yann LeCun’s Bet Against Large Language Models
Quantum Zeitgeist · 17 iul. 2026
LeRobot v0.6.0: Imagine, Evaluate, Improve
Hugging Face · 7 iul. 2026