aiqbench analysis
VersualRL introduces a closed-loop framework that leverages frozen vision-language and large language models to iteratively refine robot Behavior Trees using visual feedback. By updating policies at the symbolic level without online gradient optimization, this approach ensures transparent reasoning and human-interpretable evolution for physical robotic tasks. The method demonstrates successful adaptation to execution failures in real-world mobile robots performing complex navigation and manipulation sequences.
VersualRL: Closed-Loop Verbal Reinforcement Learning with Visual Execution Feedback for Task-Level Robot Planning
arXiv:2603.22169v3 Announce Type: replace Abstract: We introduce VersualRL, a closed-loop framework for task-level robot planning that uses visual execution feedback to iteratively refine executable Behavior Trees through structured natural-language critique. VersualRL combines a vision-language model critic, which analyzes visual observations and Behavior Tree execution traces, with a large language model actor that performs discrete and interpretable policy updates. During physical deployment, both foundation models remain frozen, while the executed Behavior Tree is updated at the symbolic level without online gradient-based policy optimization. This enables transparent reasoning, explicit causal feedback, and human-interpretable policy evolution. We validate VersualRL on a real mobile robot performing a multi-stage navigation and manipulation task under execution uncertainty. Experimental results show that the framework supports explainable policy improvements, closed-loop adaptation to execution failures, and successful deployment on physical robotic systems.
Abstract:We introduce VersualRL, a closed-loop framework for task-level robot planning that uses visual execution feedback to iteratively refine executable Behavior Trees through structured natural-language critique. VersualRL combines a vision-language model critic, which analyzes visual observations and Behavior Tree execution traces, with a large language model actor that performs discrete and interpretable policy updates. During physical deployment, both foundation models remain frozen, while the executed Behavior Tree is updated at the symbolic level without online gradient-based policy optimization. This enables transparent reasoning, explicit causal feedback, and human-interpretable policy evolution. We validate VersualRL on a real mobile robot performing a multi-stage navigation and manipulation task under execution uncertainty. Experimental results show that the framework supports explainable policy improvements, closed-loop adaptation to execution failures, and successful deployment on physical robotic systems.
Robotics (cs.RO)
arXiv:2603.22169 [cs.RO]
(or arXiv:2603.22169v3 [cs.RO] for this version)
Full text under the publisher's open licence · 138 words
Related coverage
The J-Space Debate, Agent Swarms, and Pacing Frontier AI - Digital Minds Newsletter #4
LessWrong · Sep 18, 2026
Closing the Feedback Loop: From Experience Extraction to Insight Governance in Verbal Reinforcement Learning
arXiv · Jun 17, 2026
Latent reasoning architectures would likely undermine CoT, our strongest oversight tool
AI Alignment Forum · Sep 23, 2026
[AINews] Opus 5.5 is good at explainer videos
Latent Space · Sep 29, 2026