analiza aiqbench
This paper tests whether vision-language models need to re-read raw images at every step of a robot task. It proposes a 3D geometric abstraction that converts scene geometry into symbolic references and compact text checkpoints, with images queried only when needed. For readers, the practical question is whether explicit structured state can reduce costly multimodal queries in agentic robot control, though the results are reported on a workshop-stage preprint and need independent replication.
Do Agentic Policies Need to Look at Every Step? Geometric Abstraction for Reasoning and Action
arXiv:2611.05058v1 Announce Type: cross Abstract: Recent multimodal language models can reason directly about robot scenes, but it is unclear whether they need to repeatedly reinterpret raw visual observations at every step of an interaction. We propose shifting routine spatial reasoning from images to text and symbols through a 3D geometric abstraction that is refreshed locally from observations and read directly by the language models. Specifically, our interface represents scene geometry via typed, evidence-aware symbolic references to object parts, grounds them from calibrated RGB-D, and returns compact textual checkpoints with metric residuals after each action. Symbolic task structure can thus be reused across decisions while metric values are rebound to the current scene, with direct visual reasoning invoked only on demand. Across offline policy synthesis and continuous online execution, reasoning over grounded text and symbols improves multi-step manipulation completion while substantially reducing repeated multimodal queries, positioning explicit geometric state as an effective substrate for agentic robot policies.
Title:Do Agentic Policies Need to Look at Every Step? Geometric Abstraction for Reasoning and Action
Abstract:Recent multimodal language models can reason directly about robot scenes, but it is unclear whether they need to repeatedly reinterpret raw visual observations at every step of an interaction. We propose shifting routine spatial reasoning from images to text and symbols through a 3D geometric abstraction that is refreshed locally from observations and read directly by the language models. Specifically, our interface represents scene geometry via typed, evidence-aware symbolic references to object parts, grounds them from calibrated RGB-D, and returns compact textual checkpoints with metric residuals after each action. Symbolic task structure can thus be reused across decisions while metric values are rebound to the current scene, with direct visual reasoning invoked only on demand. Across offline policy synthesis and continuous online execution, reasoning over grounded text and symbols improves multi-step manipulation completion while substantially reducing repeated multimodal queries, positioning explicit geometric state as an effective substrate for agentic robot policies.
Accepted to the CoRL 2026 Workshop on Agentic Robotics. 13 pages, 8 figures. Project page: this https URL
Robotics (cs.RO); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
arXiv:2611.05058 [cs.RO]
(or arXiv:2611.05058v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2611.05058 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 286 cuvinte
Subiecte conexe
Neuro-Symbolic Geometric Abstraction (NeuSOGA): From Observations to Symbolic Mathematical Representations
arXiv · 2 sept. 2026
Local Geometric Mixing via Dobrushin Contraction with Applications to Diffusion Path Monte Carlo and the Proximal Sampler
arXiv · 24 sept. 2026
NeuSOGA3D: A Neuro-Symbolic Framework for Explainable 3D Geometric Reconstruction
arXiv · 18 sept. 2026
CReF: Cross-modal and Recurrent Fusion for Depth-conditioned Humanoid Locomotion
arXiv · 28 iul. 2026