aiqbench analysis
This research introduces SGImagineNav, a framework that uses large vision language models and symbolic scene graphs to enable embodied agents to navigate unseen environments by predicting future states before acting. By proactively building global environmental representations, the system achieves superior success rates in both simulation and real-world cross-floor navigation benchmarks compared to observation-only methods.
Imaginative World Modeling with Scene Graphs for Embodied Agent Navigation
arXiv:2508.06990v2 Announce Type: replace Abstract: Semantic navigation requires an agent to navigate toward a specified target in an unseen environment. Employing an imaginative navigation strategy that predicts future scenes before taking action, can empower the agent to find target faster. Inspired by this idea, we propose SGImagineNav, a novel imaginative navigation framework that leverages symbolic world modeling to proactively build a global environmental representation. SGImagineNav maintains an evolving hierarchical scene graph and uses large vision language models to predict and explore unseen parts of the environment. While existing methods solely relying on past observations, this imaginative scene graph provides richer semantic context, enabling the agent to proactively estimate target locations. Building upon this, SGImagineNav adopts an adaptive navigation strategy that exploits semantic shortcuts when promising and explores unknown areas otherwise to gather additional context. This strategy continuously expands the known environment and accumulates valuable semantic contexts, ultimately guiding the agent toward the target. SGImagineNav is evaluated in both real-world scenarios and simulation benchmarks. SGImagineNav consistently outperforms previous methods, improving the success rate to 65.4% and 66.8% on HM3D and HSSD, and demonstrating cross-floor and cross-room navigation in real-world environments. All source code is open-sourced at https://github.com/UMich-CURLY/SGImagineNav.
Abstract:Semantic navigation requires an agent to navigate toward a specified target in an unseen environment. Employing an imaginative navigation strategy that predicts future scenes before taking action, can empower the agent to find target faster. Inspired by this idea, we propose SGImagineNav, a novel imaginative navigation framework that leverages symbolic world modeling to proactively build a global environmental representation. SGImagineNav maintains an evolving hierarchical scene graph and uses large vision language models to predict and explore unseen parts of the environment. While existing methods solely relying on past observations, this imaginative scene graph provides richer semantic context, enabling the agent to proactively estimate target locations. Building upon this, SGImagineNav adopts an adaptive navigation strategy that exploits semantic shortcuts when promising and explores unknown areas otherwise to gather additional context. This strategy continuously expands the known environment and accumulates valuable semantic contexts, ultimately guiding the agent toward the target. SGImagineNav is evaluated in both real-world scenarios and simulation benchmarks. SGImagineNav consistently outperforms previous methods, improving the success rate to 65.4% and 66.8% on HM3D and HSSD, and demonstrating cross-floor and cross-room navigation in real-world environments. All source code is open-sourced at this https URL.
Robotics (cs.RO)
arXiv:2508.06990 [cs.RO]
(or arXiv:2508.06990v2 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2508.06990 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 284 words
Related coverage
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · Sep 25, 2026
Presentation: Designing AI Platforms for Reliability: Tools for Certainty, Agents for Discovery
InfoQ · May 27, 2026
HarnessWAM: Bridging Prediction and Deliberation in World Action Models
arXiv · Oct 14, 2026
EmbodiedLGR: Integrating Lightweight Graph Representation and Retrieval for Semantic-Spatial Memory in Robotic Agents
arXiv · Sep 7, 2026