RankFormer: A Propose-then-Select Transformer for Multi-Agent Multimodal Trajectory Prediction

arXiv:2604.07126v2 Announce Type: replace-cross Abstract: Predicting vehicle trajectories plays an important role in autonomous driving, transportation safety analysis, traffic operations, etc. Although many deep learning algorithms are devised to predict future vehicle trajectories, the vehicle trajectory prediction problem is still challenging due to the complexity of decision-making process, interactions with surrounding vehicles, and the existence of multiple possible intentions for the traveling agents even under similar scenarios. For modeling interactions between vehicles, previous methods are either limited by specific graph structure (e.g., Graph Neural Network) or limited by fixed labeled intentions. In this study, we propose a pure Transformer-based network considering both temporal dependencies and spatial interactions without specific graph structures or labeled samples for intentions. By employing a cross-modal attention module, the model can learn a group of trajectories with ordered intentions. Also, we enhance the spatial encoding module to consider ego-centric velocity and acceleration of neighboring vehicles. Two tracks of decoders are employed to learn the ordered group of trajectories with ordered intentions and their probabilities. The probability decoder also provides by-product of spatial attentions among traveling vehicles. In short, the proposed model provides an efficient and effective way to predict agent trajectories under aerial scenes.

Abstract:Predicting vehicle trajectories plays an important role in autonomous driving, transportation safety analysis, traffic operations, etc. Although many deep learning algorithms are devised to predict future vehicle trajectories, the vehicle trajectory prediction problem is still challenging due to the complexity of decision-making process, interactions with surrounding vehicles, and the existence of multiple possible intentions for the traveling agents even under similar scenarios. For modeling interactions between vehicles, previous methods are either limited by specific graph structure (e.g., Graph Neural Network) or limited by fixed labeled intentions. In this study, we propose a pure Transformer-based network considering both temporal dependencies and spatial interactions without specific graph structures or labeled samples for intentions. By employing a cross-modal attention module, the model can learn a group of trajectories with ordered intentions. Also, we enhance the spatial encoding module to consider ego-centric velocity and acceleration of neighboring vehicles. Two tracks of decoders are employed to learn the ordered group of trajectories with ordered intentions and their probabilities. The probability decoder also provides by-product of spatial attentions among traveling vehicles. In short, the proposed model provides an efficient and effective way to predict agent trajectories under aerial scenes.

Robotics (cs.RO); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

arXiv:2604.07126 [cs.RO]

(or arXiv:2604.07126v2 [cs.RO] for this version)

https://doi.org/10.48550/arXiv.2604.07126 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

aiqbench analysis

RankFormer: A Propose-then-Select Transformer for Multi-Agent Multimodal Trajectory Prediction

Related coverage