aiqbench analysis
This paper combines a vision-language model with control barrier functions so a robot can adjust its safety margins based on what it sees, for example keeping more distance from a human than from a static object of the same shape. The reported results are from matched-geometry scenarios and one physical robot, so generalization to other environments is still untested. The architecture is practically useful because the VLM runs asynchronously and outputs a scalar signal from token probabilities, keeping it out of the safety-critical control loop.
Semantically Adaptive Control Barrier Functions with Vision-Language Models for Real-Time Robot Navigation
arXiv:2603.21142v2 Announce Type: replace Abstract: Robots operating in dynamic, unstructured environments must maintain safety without becoming unnecessarily conservative. While control barrier functions (CBFs) provide a principled mechanism for collision avoidance, their behavior is typically determined from geometric state alone and therefore cannot distinguish between semantically different situations with similar physical configurations. We present AlphaAdj, a vision-to-control navigation framework that uses egocentric RGB observations and a vision-language model (VLM) to adapt CBF conservativeness according to visual semantic context. The VLM produces a scalar contextual caution signal that dynamically modulates the CBF parameter alpha, while a geometry- and speed-dependent cap bounds admissible aggressiveness. To support real-time operation despite VLM latency, inference runs asynchronously with the control loop, and the semantic signal is extracted directly from token-level output probabilities rather than full autoregressive text generation. We evaluate AlphaAdj across matched-geometry, dynamic-human, and cluttered navigation scenarios, as well as on a physical Clearpath Jackal. When obstacle geometry is held fixed, AlphaAdj allocates 0.405 m more clearance to a human than to a benign object, while geometry-only and non-visual adaptive CBF baselines produce no semantic differentiation. This additional behavioral reserve improves robustness to unexpected human motion, including cluttered conditions in which all evaluated geometric baselines collide while AlphaAdj remains collision-free. The system maintains a 0.0167 s control period with 0.158 s mean VLM round-trip latency, demonstrating that semantic visual reasoning can complement geometric safety control without entering the critical control path.
Abstract:Robots operating in dynamic, unstructured environments must maintain safety without becoming unnecessarily conservative. While control barrier functions (CBFs) provide a principled mechanism for collision avoidance, their behavior is typically determined from geometric state alone and therefore cannot distinguish between semantically different situations with similar physical configurations. We present AlphaAdj, a vision-to-control navigation framework that uses egocentric RGB observations and a vision-language model (VLM) to adapt CBF conservativeness according to visual semantic context. The VLM produces a scalar contextual caution signal that dynamically modulates the CBF parameter alpha, while a geometry- and speed-dependent cap bounds admissible aggressiveness. To support real-time operation despite VLM latency, inference runs asynchronously with the control loop, and the semantic signal is extracted directly from token-level output probabilities rather than full autoregressive text generation. We evaluate AlphaAdj across matched-geometry, dynamic-human, and cluttered navigation scenarios, as well as on a physical Clearpath Jackal. When obstacle geometry is held fixed, AlphaAdj allocates 0.405 m more clearance to a human than to a benign object, while geometry-only and non-visual adaptive CBF baselines produce no semantic differentiation. This additional behavioral reserve improves robustness to unexpected human motion, including cluttered conditions in which all evaluated geometric baselines collide while AlphaAdj remains collision-free. The system maintains a 0.0167 s control period with 0.158 s mean VLM round-trip latency, demonstrating that semantic visual reasoning can complement geometric safety control without entering the critical control path.
Robotics (cs.RO)
arXiv:2603.21142 [cs.RO]
(or arXiv:2603.21142v2 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2603.21142 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 322 words