aiqbench analysis
This research introduces SG-CoT, a framework that enhances robotic planning by grounding Large Language Model reasoning in structured scene graphs to resolve environmental ambiguities. By enabling models to query specific object relationships and request disambiguation from users or other agents, the method significantly improves task success rates in both single-agent and multi-agent scenarios without relying on external tools like code interpreters.
SG-CoT: An Ambiguity-Aware Robotic Planning Framework using Scene Graph Representations
arXiv:2603.18271v3 Announce Type: replace Abstract: Ambiguity poses a major challenge to large language models (LLMs) used as robotic planners. In this letter, we present Scene Graph-Chain-of-Thought (SG-CoT), a two-stage framework where LLMs iteratively query a scene graph representation of the environment to detect and clarify ambiguities. First, a structured scene graph representation of the environment is constructed from input observations, capturing objects, their attributes, and relationships with other objects. Second, the LLM is equipped with retrieval functions to query portions of the scene graph that are relevant to the provided instruction. This grounds the reasoning process of the LLM in the observation, increasing the reliability of robotic planners under ambiguous situations. SG-CoT also allows the LLM to identify the source of ambiguity and pose a relevant disambiguation question to the user or another robot. Extensive experimentation demonstrates that SG-CoT consistently outperforms prior methods, with a minimum of 10% improvement in question accuracy and a minimum success rate increase of 4% in single-agent and 15% in multi-agent environments, validating its effectiveness for more generalizable robot planning.
Abstract:Ambiguity poses a major challenge to large language models (LLMs) used as robotic planners. In this letter, we present Scene Graph-Chain-of-Thought (SG-CoT), a two-stage framework where LLMs iteratively query a scene graph representation of the environment to detect and clarify ambiguities. First, a structured scene graph representation of the environment is constructed from input observations, capturing objects, their attributes, and relationships with other objects. Second, the LLM is equipped with retrieval functions to query portions of the scene graph that are relevant to the provided instruction. This grounds the reasoning process of the LLM in the observation, increasing the reliability of robotic planners under ambiguous situations. SG-CoT also allows the LLM to identify the source of ambiguity and pose a relevant disambiguation question to the user or another robot. Extensive experimentation demonstrates that SG-CoT consistently outperforms prior methods, with a minimum of 10% improvement in question accuracy and a minimum success rate increase of 4% in single-agent and 15% in multi-agent environments, validating its effectiveness for more generalizable robot planning.
Robotics (cs.RO)
arXiv:2603.18271 [cs.RO]
(or arXiv:2603.18271v3 [cs.RO] for this version)
Full text under the publisher's open licence · 180 words
Related coverage
Links #5: 2026/07
LessWrong · Jul 31, 2026
An Information-Space Perspective to Scene Graph Sufficiency for Robotic Task Planning
arXiv · Sep 15, 2026
Embodied Zero-Shot Task Planning via Hardware-Aware LLM-VLM Integration for Grounded Robotic Manipulation
arXiv · Oct 15, 2026
HINT-Plan: Human Intention-Aware Robot Task Planning in Context-Rich Environments using Vision Language Models
arXiv · Sep 17, 2026