aiqbench analysis
The paper proposes a navigation world model that learns from large volumes of action-free video, using latent actions as a bridge before a smaller action-labeled post-training stage. The main claims are gains in visual prediction and navigation on in-domain and out-of-domain benchmarks, backed by a new 17.5-million-observation dataset. For AI readers, the key questions are how much of the gain comes from the latent-action pretraining itself versus the dataset scale, and how well it transfers to real robots outside the benchmarks.
OpenNWM: A Generalist Navigation World Model with Latent Action Pretraining
arXiv:2611.01844v1 Announce Type: cross Abstract: Navigation world models predict future observations conditioned on navigation actions to support planning, but their generalization across real-world environments remains constrained by the limited coverage of action-labeled data. Action-free navigation videos provide diverse visual experience, yet lack the recorded actions required for action-conditioned training. We introduce OpenNWM, a generalist navigation world model with latent action pretraining. It learns universal latent actions from large-scale action-free videos through visual prediction pretraining, and further aligns them with physical motions via action-conditioned post-training. To support video pretraining, we curate NavAnywhere, a large-scale action-free navigation video dataset comprising 17.5 million visual observations from 15 heterogeneous sources. It captures navigation by ground robots, humans, and drones, with diverse viewpoints and motion patterns across five broad scene types: residential areas, public and commercial spaces, urban settings, parks, and natural landscapes. Experiments demonstrate improved visual prediction on in-domain and out-of-domain benchmarks, together with improved navigation performance. These results show that latent-action pretraining enables diverse, action-free videos to improve dynamics learning and generalization for open-world navigation.
Abstract:Navigation world models predict future observations conditioned on navigation actions to support planning, but their generalization across real-world environments remains constrained by the limited coverage of action-labeled data. Action-free navigation videos provide diverse visual experience, yet lack the recorded actions required for action-conditioned training. We introduce OpenNWM, a generalist navigation world model with latent action pretraining. It learns universal latent actions from large-scale action-free videos through visual prediction pretraining, and further aligns them with physical motions via action-conditioned post-training. To support video pretraining, we curate NavAnywhere, a large-scale action-free navigation video dataset comprising 17.5 million visual observations from 15 heterogeneous sources. It captures navigation by ground robots, humans, and drones, with diverse viewpoints and motion patterns across five broad scene types: residential areas, public and commercial spaces, urban settings, parks, and natural landscapes. Experiments demonstrate improved visual prediction on in-domain and out-of-domain benchmarks, together with improved navigation performance. These results show that latent-action pretraining enables diverse, action-free videos to improve dynamics learning and generalization for open-world navigation.
Robotics (cs.RO); Artificial Intelligence (cs.AI)
arXiv:2611.01844 [cs.RO]
(or arXiv:2611.01844v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2611.01844 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 263 words
Related coverage
Robot Learning with Vision-Language-Action Models for Agricultural Manipulation
arXiv · Oct 20, 2026
Benchmarking Generalist and Specialist Time-Series Foundation Models for PPG-Based Health Assessment
arXiv · Nov 3, 2026
Do as the Romans Do: Learning Universal Behaviors from Heterogeneous Agents
arXiv · Oct 5, 2026
PINT: A Unified Model with Shared Predictive Intent for Robot Manipulation
arXiv · Oct 14, 2026