analiza aiqbench
DepthEvidence advances multimodal language models by integrating dense metric depth predictions directly into the generation process, enabling robust geometric reasoning with numerical constraints. This approach bridges the gap between specialized vision estimators and general-purpose LLMs, offering a unified framework for spatial understanding that preserves object-grounded evidence during language interaction. The introduction of a new Depth-VQA benchmark further validates these capabilities across diverse datasets, setting a new standard for evaluating metric depth in AI systems.
DepthEvidence: Unifying Metric Depth Prediction and Geometric Reasoning in Multimodal Language Models
arXiv:2609.31103v1 Announce Type: cross Abstract: Spatial reasoning with metric constraints requires linking objects to geometric measurements and preserving their numerical content during language reasoning. We present DepthEvidence, a 4B model that uses its own dense metric predictions as object-grounded evidence for language generation. A camera-conditioned decoder predicts full-resolution metric depth using multi-scale visual features and high-resolution RGB refinement. A dense-to-language interface converts predicted depths and decoder features into object-aligned continuous geometry tokens anchored to object identifiers. Geometric supervision encourages metric information to remain recoverable before and after language-context interaction, while instruction tuning supports object measurement and compositional reasoning. We introduce a Depth-VQA benchmark evaluating object-depth queries, relative comparisons, and decisions combining spatial and numerical constraints. Across nine datasets, DepthEvidence achieves the highest average dense $\delta_1$ among evaluated methods, competitive with specialized estimators. It also leads the evaluated methods in instance-level metric depth estimation and overall accuracy on both relative and metric reasoning tracks, while broadly preserving general VQA performance and improving spatial understanding relative to the base model.
Abstract:Spatial reasoning with metric constraints requires linking objects to geometric measurements and preserving their numerical content during language reasoning. We present DepthEvidence, a 4B model that uses its own dense metric predictions as object-grounded evidence for language generation. A camera-conditioned decoder predicts full-resolution metric depth using multi-scale visual features and high-resolution RGB refinement. A dense-to-language interface converts predicted depths and decoder features into object-aligned continuous geometry tokens anchored to object identifiers. Geometric supervision encourages metric information to remain recoverable before and after language-context interaction, while instruction tuning supports object measurement and compositional reasoning. We introduce a Depth-VQA benchmark evaluating object-depth queries, relative comparisons, and decisions combining spatial and numerical constraints. Across nine datasets, DepthEvidence achieves the highest average dense $\delta_1$ among evaluated methods, competitive with specialized estimators. It also leads the evaluated methods in instance-level metric depth estimation and overall accuracy on both relative and metric reasoning tracks, while broadly preserving general VQA performance and improving spatial understanding relative to the base model.
Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
arXiv:2609.31103 [cs.CV]
(or arXiv:2609.31103v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2609.31103 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 263 cuvinte
Subiecte conexe
Thinker-WM: A Unified World Model with Shared Multimodal Representations for Embodied Intelligence
arXiv · 14 oct. 2026
Learning Geometric Representations from Videos for Spatial Intelligent Multimodal Large Language Models
arXiv · 6 iun. 2026
AI customer service: strategy, agents, and solutions guide
Databricks · 27 iul. 2026
PhysWAM: Physically Consistent World Action Model for Autonomous Driving
arXiv · 30 sept. 2026