analiza aiqbench
VLGA addresses a critical gap in autonomous driving by treating geometry as a distinct fourth modality, enabling VLA models to reconstruct dense 3D environments directly from LiDAR data rather than relying on sparse constraints. This approach yields state-of-the-art performance on nuScenes and Bench2Drive benchmarks, significantly reducing collision rates and improving driving scores while maintaining efficiency. The work represents a substantive methodological shift for the robotics community, moving beyond frozen foundation models toward policies that are explicitly supervised for dense spatial understanding.
VLGA: Vision-Language-Geometry-Action Models for Autonomous Driving
arXiv:2606.12396v1 Announce Type: cross Abstract: Vision-language-action (VLA) models can describe scenes and reason about them in language, yet still struggle to ground their actions in the dense 3D world around them. Existing approaches either inject features from a frozen 3D foundation model without an objective that ensures the policy uses them, or constrain geometry with sparse box and map losses that provide no dense spatial signal. We introduce VLGA, the first vision-language-action model supervised to reconstruct the dense 3D world it drives through. VLGA introduces geometry as a fourth modality alongside vision, language, and action through a dedicated expert supervised by a per-pixel pointmap regression loss against LiDAR. Extensive experiments conducted on challenging nuScenes and Bench2Drive datasets for open-loop and closed-loop evaluations, respectively, show the superiority of VLGA over counterpart VLA methods. In particular, on open-loop nuScenes, VLGA sets a new state of the art among VLA methods without ego status, with the lowest L2 (0.50\,m average) and 3-second collision rate (0.18\%). On closed-loop Bench2Drive, VLGA attains the state-of-the-art driving score of 79.08, +0.71 over the strongest prior VLA, at comparable efficiency and comfort.
Abstract:Vision-language-action (VLA) models can describe scenes and reason about them in language, yet still struggle to ground their actions in the dense 3D world around them. Existing approaches either inject features from a frozen 3D foundation model without an objective that ensures the policy uses them, or constrain geometry with sparse box and map losses that provide no dense spatial signal. We introduce VLGA, the first vision-language-action model supervised to reconstruct the dense 3D world it drives through. VLGA introduces geometry as a fourth modality alongside vision, language, and action through a dedicated expert supervised by a per-pixel pointmap regression loss against LiDAR. Extensive experiments conducted on challenging nuScenes and Bench2Drive datasets for open-loop and closed-loop evaluations, respectively, show the superiority of VLGA over counterpart VLA methods. In particular, on open-loop nuScenes, VLGA sets a new state of the art among VLA methods without ego status, with the lowest L2 (0.50\,m average) and 3-second collision rate (0.18\%). On closed-loop Bench2Drive, VLGA attains the state-of-the-art driving score of 79.08, +0.71 over the strongest prior VLA, at comparable efficiency and comfort.
Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
arXiv:2606.12396 [cs.CV]
(or arXiv:2606.12396v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2606.12396 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 278 cuvinte
Subiecte conexe
Geo-VLA: Geometry-Aware Vision-Language-Action Planning via Internalization of Map Semantics
arXiv · 25 aug. 2026
Generate Trajectories, Reasoning Traces, and Auto-Labels with NVIDIA Alpamayo 2 Super
NVIDIA · 4 aug. 2026
NVIDIA Enables the Next Era Of Physical AI Research With Agent Skills For Autonomous Vehicles, Robotics And Vision AI
NVIDIA · 3 iun. 2026
VGGTWorld-VLA: Intent-Conditioned 3D World Evolution for Autonomous Driving
arXiv · 9 oct. 2026