aiqbench analysis
This research demonstrates a significant convergence of AI and quantum physics by applying multi-modal foundation models to streamline high-energy physics reconstruction chains. By treating diverse physical objects like tracks and jets as tokens within a shared vocabulary, the study enables a single model to perform various tasks—ranging from detector simulation to particle flow—with high fidelity. This approach offers a promising pathway for reducing computational overhead in collider experiments while maintaining interpretability through intermediate object generation.
Prompting Particle Physics: Tokenized Multi-modal Foundation Models for Combinatorially Many Tasks
arXiv:2609.31862v1 Announce Type: cross Abstract: Reconstruction and simulation at a collider experiment are long chains of specialised algorithms, each tuned to a single step. We explore how one model can serve many of those steps at once, while still producing the intermediate objects (tracks, calorimeter cells, clusters, particles and jets) that make the chain interpretable. To do so, we represent every object in a jet in a single shared token vocabulary and train one model to map any subset of these modalities to any other. A task is then only a choice of which modalities to provide and which to request: particle flow, detector simulation and charged energy subtraction are all directions through the same set of weights. We train over all modality combinations, with a decoder emitting tokens either autoregressively or in parallel. With tokenisation, both architectures train stably with little tuning. In evaluations, both models produce realistic reconstruction and simulation objects, with the autoregressive model particularly faithful to output from Geant4. The autoregressive model also outperforms a state-of-the-art particle flow algorithm on many typical jet reconstruction metrics.
Abstract:Reconstruction and simulation at a collider experiment are long chains of specialised algorithms, each tuned to a single step. We explore how one model can serve many of those steps at once, while still producing the intermediate objects (tracks, calorimeter cells, clusters, particles and jets) that make the chain interpretable. To do so, we represent every object in a jet in a single shared token vocabulary and train one model to map any subset of these modalities to any other. A task is then only a choice of which modalities to provide and which to request: particle flow, detector simulation and charged energy subtraction are all directions through the same set of weights. We train over all modality combinations, with a decoder emitting tokens either autoregressively or in parallel. With tokenisation, both architectures train stably with little tuning. In evaluations, both models produce realistic reconstruction and simulation objects, with the autoregressive model particularly faithful to output from Geant4. The autoregressive model also outperforms a state-of-the-art particle flow algorithm on many typical jet reconstruction metrics.
47 pages. Code: this https URL ; data and models: this https URL
High Energy Physics - Phenomenology (hep-ph); Machine Learning (cs.LG); High Energy Physics - Experiment (hep-ex); Data Analysis, Statistics and Probability (physics.data-an)
arXiv:2609.31862 [hep-ph]
(or arXiv:2609.31862v1 [hep-ph] for this version)
https://doi.org/10.48550/arXiv.2609.31862 arXiv-issued DOI via DataCite (pending registration)
Full text under the publisher's open licence · 223 words
Related coverage
[AINews] The Future of Latent Space
Latent Space · Sep 25, 2026
[AINews] Opus 5.5 is good at explainer videos
Latent Space · Sep 29, 2026
Welcome NVIDIA Cosmos 3: The First Open Omni-model for Physical AI Reasoning and Action
Hugging Face · Jun 1, 2026
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · Sep 25, 2026