aiqbench analysis
LookStep introduces a resource-efficient framework for Vision-Language Navigation by leveraging Multimodal Large Language Models to predict future states and manage memory through semantic event labeling. This approach significantly reduces computational overhead and data requirements compared to traditional methods that rely on heavy cognitive maps or external 3D tools. The study demonstrates that autonomous, language-driven state management can achieve superior performance in unseen environments while maintaining bounded memory usage.
LookStep: Efficient Vision-Language Navigation with Linguistic Foresight and Event Driven Memory
arXiv:2609.02350v1 Announce Type: cross Abstract: Vision-Language Navigation (VLN) requires an embodied agent to follow natural-language instructions in unseen environments. Recent progress has been largely driven by Multimodal Large Language Models (MLLMs). Existing methods follow a next-step action prediction paradigm, supervising only the expert action, which requires a high quantity of data for training. They also rely on cognitive maps, accumulated historical frames, or external 3D tools to maintain states, leading to high computational and memory overhead. To realize resource efficiency VLN, we propose LookStep, a unified end-to-end framework that combines Language Centric Future State Modeling and Event Driven Rolling Memory that uses language labels to generate coarse-grained navigation progress and future states for each candidate action, while autonomously deciding whether to write each observation into a bounded rolling memory with a semantic role. We validate LookStep empirically. On VLN-CE tasks, LookStep outperforms existing methods under the same training settings, achieving a 49.7\% success rate on R2R-CE Val-Unseen with better memory efficiency and less data usage. Code and model is available at https://github.com/kunyang-YU/LookStep.
Abstract:Vision-Language Navigation (VLN) requires an embodied agent to follow natural-language instructions in unseen environments. Recent progress has been largely driven by Multimodal Large Language Models (MLLMs). Existing methods follow a next-step action prediction paradigm, supervising only the expert action, which requires a high quantity of data for training. They also rely on cognitive maps, accumulated historical frames, or external 3D tools to maintain states, leading to high computational and memory overhead. To realize resource efficiency VLN, we propose LookStep, a unified end-to-end framework that combines Language Centric Future State Modeling and Event Driven Rolling Memory that uses language labels to generate coarse-grained navigation progress and future states for each candidate action, while autonomously deciding whether to write each observation into a bounded rolling memory with a semantic role. We validate LookStep empirically. On VLN-CE tasks, LookStep outperforms existing methods under the same training settings, achieving a 49.7\% success rate on R2R-CE Val-Unseen with better memory efficiency and less data usage. Code and model is available at this https URL.
19 Pages, 7 Figures. Accepted in EMNLP 2026 Main
Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
arXiv:2609.02350 [cs.CV]
(or arXiv:2609.02350v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2609.02350 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 277 words
Related coverage
Token-Wise Latent Streaming from Slow Reasoners to Fast Planners for Dynamic Vision Language Navigation
arXiv · Jul 21, 2026
The Future of Aviation Management: Reimagining the Aviation MBA
slguardian.org · Aug 6, 2026
PreAct-Nav: Agentic Reasoning Before Action for Urban Navigation
arXiv · Oct 7, 2026
Tiny Aya in the wild: What people are building with open multilingual AI
Cohere · Jul 14, 2026