analiza aiqbench
This study critically evaluates the reliability of current video-language models in safety-critical driving scenarios, revealing that standard prompting techniques often fail to capture robust temporal dynamics compared to lightweight LoRA fine-tuning or intermediate feature probes. The findings indicate that existing models frequently act as frame-level recognizers with inherent biases rather than dependable temporal world models, highlighting a significant gap between general-purpose claims and performance in motion-discriminative tasks.
Mind the Temporal Gap: Evaluating Video Models for Temporal Understanding in Driving
arXiv:2610.12575v1 Announce Type: cross Abstract: Modern video-language models are increasingly treated as general-purpose video reasoners, yet it remains unclear whether they robustly capture temporal dynamics in safety-critical settings such as driving. We study this question through controlled ego-motion tasks that require identifying actions from driving videos. Across video-language models, we find that standard prompting, visual prompting, few-shot prompting, and chain-of-thought prompting remain unreliable and highly task- and model-dependent, with persistent class biases, although newer models show substantial task-specific few-shot gains. In contrast, lightweight LoRA finetuning and simple probes trained on intermediate visual features recover strong motion-discriminative performance, indicating that motion-relevant cues are often present inside the models but are not reliably used by the final video-to-language pathway. These results suggest that, in driving, current video-language models remain unreliable temporal reasoners and often behave more like frame-level, bias-driven recognizers than dependable temporal world models.
Abstract:Modern video-language models are increasingly treated as general-purpose video reasoners, yet it remains unclear whether they robustly capture temporal dynamics in safety-critical settings such as driving. We study this question through controlled ego-motion tasks that require identifying actions from driving videos. Across video-language models, we find that standard prompting, visual prompting, few-shot prompting, and chain-of-thought prompting remain unreliable and highly task- and model-dependent, with persistent class biases, although newer models show substantial task-specific few-shot gains. In contrast, lightweight LoRA finetuning and simple probes trained on intermediate visual features recover strong motion-discriminative performance, indicating that motion-relevant cues are often present inside the models but are not reliably used by the final video-to-language pathway. These results suggest that, in driving, current video-language models remain unreliable temporal reasoners and often behave more like frame-level, bias-driven recognizers than dependable temporal world models.
Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
arXiv:2610.12575 [cs.CV]
(or arXiv:2610.12575v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2610.12575 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 237 cuvinte
Subiecte conexe
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · 25 sept. 2026
From Segments to Scenes: Temporal Understanding for Agentic Autonomous Driving via Vision-Language Models
arXiv · 4 iun. 2026
NVIDIA Nemotron 3 Nano Omni Powers Multimodal Agent Reasoning in a Single Efficient Open Model
NVIDIA · 28 apr. 2026
Underwriting Superintelligence: Backing Agents you can Sue — Rune Kvist, AIUC
Latent Space · 16 sept. 2026