analiza aiqbench
This research addresses a core limitation in Vision-Language Models by introducing Geometry-Privileged Distillation (GPD), which leverages 3D geometric cues during training to improve spatial reasoning without altering the inference-time architecture or latency. The method routes depth and bird's-eye-view information to a teacher model specifically for correcting incorrect trajectories, achieving significant performance gains on benchmarks like VSI-Bench while keeping the deployed model RGB-only.
Distilling Routed 3D Privilege for Spatial Reasoning in Vision-Language Models
arXiv:2610.12355v1 Announce Type: cross Abstract: Spatial reasoning remains a persistent weakness of vision-language models (VLMs), because RGB inputs do not directly provide geometric evidence. Existing remedies either inject 3D into the model at inference, paying architecture and latency costs, or train with outcome rewards that supervise only the final answer. Spatial errors originate in perception: a misjudged depth or direction can be corrected only by the scene's true geometry, which the 3D-scanned sources of spatial training corpora already provide. We propose GPD (Geometry-Privileged Distillation), which makes geometric evidence the privilege in on-policy self-distillation (OPSD). For each question, depth, semantic, and bird's-eye-view (BEV) cues are rendered as compact text and routed to the teacher alongside the reference answer; a privileged KL, applied only to incorrect trajectories, augments GRPO, and the deployed model remains RGB-only. On the 4B backbone, GPD achieves 57.1 on VSI-Bench and 37.6 average across MindCube, SPARBench, MMSI-Bench, and ViewSpatial, outperforming both GRPO and answer-privileged OPSD across spatial reasoning benchmarks. Ablations confirm the complementarity of 3D and answer privilege, the advantage of question-conditioned routing over full-context injection, and the benefit of restricting distillation to incorrect trajectories.
Abstract:Spatial reasoning remains a persistent weakness of vision-language models (VLMs), because RGB inputs do not directly provide geometric evidence. Existing remedies either inject 3D into the model at inference, paying architecture and latency costs, or train with outcome rewards that supervise only the final answer. Spatial errors originate in perception: a misjudged depth or direction can be corrected only by the scene's true geometry, which the 3D-scanned sources of spatial training corpora already provide. We propose GPD (Geometry-Privileged Distillation), which makes geometric evidence the privilege in on-policy self-distillation (OPSD). For each question, depth, semantic, and bird's-eye-view (BEV) cues are rendered as compact text and routed to the teacher alongside the reference answer; a privileged KL, applied only to incorrect trajectories, augments GRPO, and the deployed model remains RGB-only. On the 4B backbone, GPD achieves 57.1 on VSI-Bench and 37.6 average across MindCube, SPARBench, MMSI-Bench, and ViewSpatial, outperforming both GRPO and answer-privileged OPSD across spatial reasoning benchmarks. Ablations confirm the complementarity of 3D and answer privilege, the advantage of question-conditioned routing over full-context injection, and the benefit of restricting distillation to incorrect trajectories.
Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
arXiv:2610.12355 [cs.CV]
(or arXiv:2610.12355v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2610.12355 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 282 cuvinte
Subiecte conexe
[AINews] not much happened today
Latent Space · 10 sept. 2026
Learning Visual Spatial Planning from Symbolic State via Modality-Gap-Aware Self-Distillation
arXiv · 9 iun. 2026
Learning Visual Spatial Planning from Symbolic State via Modality-Gap-Aware Self-Distillation
arXiv · 6 iun. 2026
Where-OPD: Spatially Guided On-Policy Self-Distillation of MLLMs with Synthetic Scenes
arXiv · 2 oct. 2026