aiqbench analysis
This research addresses a critical limitation in latent world models like LeWM, where standard Euclidean distance metrics fail to accurately reflect environmental proximity outside the immediate goal neighborhood. By proposing the Neighborhood Graph Cost (NGC) to measure distances along the underlying latent manifold, the study significantly improves planning success rates without requiring model retraining. This advancement demonstrates that respecting the geometric structure of latent spaces is essential for robust robotic planning and world modeling.
Latent World Models Should Plan with Distances Along the Latent Manifold
arXiv:2610.25013v1 Announce Type: new Abstract: Latent world models such as LeWM plan by searching for an action sequence whose predicted terminal latent is closest to the goal latent under an ambient norm, such as the Euclidean distance. Planning with such a cost assumes that proximity in the ambient latent space reflects proximity to the goal in the environment. Studying LeWM, we find that this assumption holds only in a small neighborhood around the goal, whereas farther away the L2 cost forms spurious low-cost basins that trap the planner. Yet neighborhood-based embeddings show that the latents retain the underlying physical state. We therefore hypothesize that the latents lie near a curved, low-dimensional manifold, and that the terminal cost should measure the distance along this latent manifold rather than through the ambient space. To estimate this manifold distance, we propose the Neighborhood Graph Cost (NGC), which computes shortest-path distances on a mutual nearest-neighbor graph constructed over training latents. NGC changes only the terminal cost and requires no retraining. This single change improves the planning success rate of LeWM across all four benchmark tasks and nearly solves TwoRoom and Reacher: even at the farthest goal offsets tested, NGC reaches 95% and 100% success, whereas the L2 cost falls to 27% and 81%. These results support the manifold hypothesis and our argument that latent world models should plan with distances along the latent manifold.
Abstract:Latent world models such as LeWM plan by searching for an action sequence whose predicted terminal latent is closest to the goal latent under an ambient norm, such as the Euclidean distance. Planning with such a cost assumes that proximity in the ambient latent space reflects proximity to the goal in the environment. Studying LeWM, we find that this assumption holds only in a small neighborhood around the goal, whereas farther away the L2 cost forms spurious low-cost basins that trap the planner. Yet neighborhood-based embeddings show that the latents retain the underlying physical state. We therefore hypothesize that the latents lie near a curved, low-dimensional manifold, and that the terminal cost should measure the distance along this latent manifold rather than through the ambient space. To estimate this manifold distance, we propose the Neighborhood Graph Cost (NGC), which computes shortest-path distances on a mutual nearest-neighbor graph constructed over training latents. NGC changes only the terminal cost and requires no retraining. This single change improves the planning success rate of LeWM across all four benchmark tasks and nearly solves TwoRoom and Reacher: even at the farthest goal offsets tested, NGC reaches 95% and 100% success, whereas the L2 cost falls to 27% and 81%. These results support the manifold hypothesis and our argument that latent world models should plan with distances along the latent manifold.
Machine Learning (cs.LG); Robotics (cs.RO)
arXiv:2610.25013 [cs.LG]
(or arXiv:2610.25013v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.25013 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 319 words
Related coverage
Distance Marching for Generative Modeling
arXiv · Oct 14, 2026
RW-Flow: One-Step Generation on Compact Manifolds via Wasserstein Gradient Flows
arXiv · Oct 1, 2026
Hyperbolic color codes with constant rate and polynomial distance
arXiv · Sep 16, 2026
A sharp gradient-corrected Lieb--Thirring inequality in one dimension with applications to density functional theory
arXiv · Oct 23, 2026