analiza aiqbench
This research introduces ViewSuite to evaluate how Vision-Language Models can plan multi-step camera trajectories in 3D environments. The study reveals a significant gap where current models struggle to compose single-view actions into coherent long-term plans but demonstrates that iterative self-exploration combined with view graph distillation effectively bridges this divide. By reshaping policy distributions through distilled exploration data, the approach enables frontier VLMs like Qwen2.5-VL-7B to achieve substantial performance gains in interactive spatial reasoning tasks.
Planning with the Views
arXiv:2605.29563v3 Announce Type: replace-cross Abstract: Can VLMs predict how each camera move changes the view, and plan many such moves ahead? We call this capability view planning, requiring (1)understanding how a single action transforms the view, and (2)composing many such transformations across multi-turn plans to identify a target view. We probe both abilities in our proposed ViewSuite, a 3D point-cloud environment on real ScanNet scenes. Across 13 frontier VLMs, a critical planning gap emerges: they possess basic view-action knowledge but fail to compose it across multi-turn plans, with the gap widening as viewpoint distance grows. To close this gap, we propose an iterative framework that alternates self-exploration with view graph distillation. The key insight is that all exploration trajectories, regardless of their outcome, collectively form a view graph that compactly captures how viewpoints connect across a scene. Distilling this graph into diverse supervised tasks reshapes the policy distribution and overcomes the sparse rewards that stall pure RL. This improves Qwen2.5-VL-7B from 2.5% to 47.8% on interactive view planning, surpassing GPT-5.4 Pro (18.5%) and Gemini 3.1 Pro (21.4%). Self-exploration emerges as a promising path toward VLMs that can actively reason and plan in 3D space. Code and Data are at https://viewsuite.github.io.
Abstract:Can VLMs predict how each camera move changes the view, and plan many such moves ahead? We call this capability view planning, requiring (1)understanding how a single action transforms the view, and (2)composing many such transformations across multi-turn plans to identify a target view. We probe both abilities in our proposed ViewSuite, a 3D point-cloud environment on real ScanNet scenes. Across 13 frontier VLMs, a critical planning gap emerges: they possess basic view-action knowledge but fail to compose it across multi-turn plans, with the gap widening as viewpoint distance grows. To close this gap, we propose an iterative framework that alternates self-exploration with view graph distillation. The key insight is that all exploration trajectories, regardless of their outcome, collectively form a view graph that compactly captures how viewpoints connect across a scene. Distilling this graph into diverse supervised tasks reshapes the policy distribution and overcomes the sparse rewards that stall pure RL. This improves Qwen2.5-VL-7B from 2.5% to 47.8% on interactive view planning, surpassing GPT-5.4 Pro (18.5%) and Gemini 3.1 Pro (21.4%). Self-exploration emerges as a promising path toward VLMs that can actively reason and plan in 3D space. Code and Data are at this https URL.
Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
arXiv:2605.29563 [cs.AI]
(or arXiv:2605.29563v3 [cs.AI] for this version)
Text integral sub licența deschisă a publicației · 216 cuvinte
Subiecte conexe
Our response to Séb Krier on Plan A
LessWrong · 14 iul. 2026
Planning with the Views via Scene Self-Exploration
arXiv · 16 iun. 2026
The Entangled Dimensions of Decision Theory
LessWrong · 30 iul. 2026
Connecting retail demand planning to campaign and store execution
Databricks · 21 aug. 2026