analiza aiqbench
GraphPoint advances robot manipulation by bridging semantic entity graphs with geometric control, enabling policies to generalize instructions through compositional reasoning rather than fixed visual mappings. This approach addresses a critical bottleneck in embodied AI where models often fail to adapt to new scenarios outside their training demonstrations. The release of the CoMani benchmark alongside the code provides a standardized framework for evaluating instruction-dependent generalization in long-horizon tasks.
GraphPoint: Semantic Entity Graphs and Point Trajectories for Compositional Robot Manipulation
arXiv:2609.18358v1 Announce Type: new Abstract: Robot manipulation policies often struggle to generalize beyond their demonstrations, even when new instructions involve familiar objects and behaviors. When language and scenes are strongly correlated during training, a policy can learn a fixed visual-action mapping rather than respond to the requested behavior. We investigate compositional reuse at two levels: within a subtask, combining familiar entities, action types, and action modifiers; and across subtasks, reusing learned subtasks in unseen long-horizon tasks. We introduce CoMani, a benchmark with controlled splits for evaluating both capabilities. Matched initial scenes and controlled changes to a single semantic factor encourage reliance on language rather than visual shortcuts. We further propose GraphPoint, which connects semantic entity graphs to geometric control by predicting future gripper point trajectories and converting them into actions using robot geometry. The framework organizes the gripper and objects by semantic roles and conditions their interactions on action types and modifiers, while predicted progress guides transitions during execution. Experiments and ablations on CoMani validate the effectiveness of our method for instruction-dependent generalization at both levels. Code will be released at GraphPoint.
Abstract:Robot manipulation policies often struggle to generalize beyond their demonstrations, even when new instructions involve familiar objects and behaviors. When language and scenes are strongly correlated during training, a policy can learn a fixed visual-action mapping rather than respond to the requested behavior. We investigate compositional reuse at two levels: within a subtask, combining familiar entities, action types, and action modifiers; and across subtasks, reusing learned subtasks in unseen long-horizon tasks. We introduce CoMani, a benchmark with controlled splits for evaluating both capabilities. Matched initial scenes and controlled changes to a single semantic factor encourage reliance on language rather than visual shortcuts. We further propose GraphPoint, which connects semantic entity graphs to geometric control by predicting future gripper point trajectories and converting them into actions using robot geometry. The framework organizes the gripper and objects by semantic roles and conditions their interactions on action types and modifiers, while predicted progress guides transitions during execution. Experiments and ablations on CoMani validate the effectiveness of our method for instruction-dependent generalization at both levels. Code will be released at GraphPoint.
Robotics (cs.RO)
arXiv:2609.18358 [cs.RO]
(or arXiv:2609.18358v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2609.18358 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 270 cuvinte
Subiecte conexe
Open-Set 3D Scene Graphs for Field Robotics: An Outdoor Case Study
arXiv · 7 sept. 2026
SGM-SLAM: Scene Graph Matching for Data-Efficient Distributed SLAM
arXiv · 16 iun. 2026
Vision-TL-Action: Neuro-Symbolic Trajectory Generation from Visual Observations and Temporal Logic
arXiv · 30 iul. 2026
Meet Harness-1: A 20B Retrieval Subagent Trained With Reinforcement Learning Inside a Stateful Search Harness on gpt-oss-20b
MarkTechPost · 7 iun. 2026