VTM-Nav: Harnessing Cross-Episode Experience for Object-Goal Navigation with Hierarchical Visual-Topological Memory

Abstract:Training-free ObjectNav agents increasingly use vision-language models (VLMs), yet typically discard acquired scene knowledge after each request. We study cross-episode ObjectNav, where each request is an independently initialized, single-goal episode and only self-acquired, scene-scoped memory persists across episodes. We ask whether an agent with fixed model parameters and navigation components can reuse such experience without retraining or oracle information. We introduce \method, a training-free framework with a persistent Hierarchical Visual-Topological Memory (VTM). VTM uses a coarse room topology to index room-owned visual memories, distinguishes in-room from remote-visible evidence, and retains successful approach cues. For each request, VTM-Nav re-localizes the agent in accumulated scene structure, retrieves target-relevant records from plausible rooms, and grounds memory guidance in candidates derived from the current observation. A conservative execution guard further handles local failures. Under matched 40-step comparisons, VTM-Nav exceeds the memory-reset WMNav control by 4.6, 2.0, and 0.8 SR points on HM3D v0.1, HM3D v0.2, and MP3D, respectively, with comparable or higher SPL. On HM3D, it also exceeds WMNav harnessed by textual memory by 3.1 and 5.5 SR points. These results demonstrate effective reuse of cross-episode scene experience through hierarchical visual-topological memory.

Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

arXiv:2607.14514 [cs.CV]

(or arXiv:2607.14514v2 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2607.14514 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.