analiza aiqbench
This research introduces MMPM, a novel framework for pedestrian trajectory prediction that explicitly separates multimodal future behaviors into distinct crossing and non-crossing modes. By incorporating gaze and gesture cues alongside vehicle interactions, the model avoids interpolating between disparate outcomes to produce semantically consistent predictions. The approach demonstrates improved accuracy on standard datasets while remaining agnostic enough to integrate with existing computer vision architectures for autonomous driving perception systems.
Where Will They Go? Modelling Multimodal Pedestrian Manoeuvres from Ego-centric Videos
arXiv:2606.18824v2 Announce Type: replace-cross Abstract: Pedestrian trajectory prediction from an on-board ego-centric camera is challenging since it depends on complex interactions with vehicles and scene context, as well as the intention of the pedestrian. The task becomes even more challenging since pedestrian intention is often ambiguous from historical observations alone, leading to an inherently multimodal distribution over future trajectories. Existing CVAE-based predictors learn a latent representation by aligning the prior conditioned on trajectory history with the posterior conditioned on both history and future trajectories. This alignment may suppress multimodality in the latent space, leading to mixed-mode predictions that interpolate between distinct future behaviours. In this paper, we propose MMPM, a mode-aware framework that separately models future trajectory distributions into semantically meaningful modes based on the pedestrian's crossing behavior. MMPM consists of two modules: behavior-aware Pedestrian Interaction Module (PIM) that jointly captures pedestrian-vehicle and pedestrian-environment interactions by introducing gaze and hand gestures, and a CVAE-based Mode-aware Trajectory Predictor (MTP) module to model the future trajectory distributions in two modes, crossing and non-crossing the road, separately. A query-based decoder further enforces mode consistency during decoding. Experiments on PIE and JAAD datasets show that our method surpasses state-of-the-art baselines. Our proposed MTP is model-agnostic, which can be integrated into existing frameworks such as BiTrap and SGNet to further improve future trajectory prediction performance. We additionally introduce a data-driven validation protocol that retrieves spatio-temporally consistent real-world trajectories from the same scene and evaluates predictions against these retrieved examples, demonstrating improved frame-wise displacement errors over previous work.
Title:Where Will They Go? Modelling Multimodal Pedestrian Manoeuvres from Ego-centric Videos
Abstract:Pedestrian trajectory prediction from an on-board ego-centric camera is challenging since it depends on complex interactions with vehicles and scene context, as well as the intention of the pedestrian. The task becomes even more challenging since pedestrian intention is often ambiguous from historical observations alone, leading to an inherently multimodal distribution over future trajectories. Existing CVAE-based predictors learn a latent representation by aligning the prior conditioned on trajectory history with the posterior conditioned on both history and future trajectories. This alignment may suppress multimodality in the latent space, leading to mixed-mode predictions that interpolate between distinct future behaviours. In this paper, we propose MMPM, a mode-aware framework that separately models future trajectory distributions into semantically meaningful modes based on the pedestrian's crossing behavior. MMPM consists of two modules: behavior-aware Pedestrian Interaction Module (PIM) that jointly captures pedestrian-vehicle and pedestrian-environment interactions by introducing gaze and hand gestures, and a CVAE-based Mode-aware Trajectory Predictor (MTP) module to model the future trajectory distributions in two modes, crossing and non-crossing the road, separately. A query-based decoder further enforces mode consistency during decoding. Experiments on PIE and JAAD datasets show that our method surpasses state-of-the-art baselines. Our proposed MTP is model-agnostic, which can be integrated into existing frameworks such as BiTrap and SGNet to further improve future trajectory prediction performance. We additionally introduce a data-driven validation protocol that retrieves spatio-temporally consistent real-world trajectories from the same scene and evaluates predictions against these retrieved examples, demonstrating improved frame-wise displacement errors over previous work.
Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
arXiv:2606.18824 [cs.CV]
(or arXiv:2606.18824v2 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2606.18824 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 356 cuvinte
Subiecte conexe
MMS-VPR: A Fine-Grained Multimodal Street-Level Visual Place Recognition Dataset and Evaluation Benchmark for Dense Pedestrian Environments
arXiv · 22 sept. 2026
Multimodal Large Language Models Predict Urban Safety Perception but Encode Non-Neutral Demographic Priors
arXiv · 1 sept. 2026
Control-Ready Uncertainty for Trajectory Diffusion
arXiv · 9 oct. 2026
NVIDIA GTC Taipei at COMPUTEX: Live Updates on What’s Next in AI
NVIDIA · 21 mai 2026