aiqbench analysis
The paper proposes a transformer variant for classifying particle jets at colliders. It reduces cost by applying full quadratic attention only to the hardest constituents and handling softer ones through linear cross-attention. The reported results are a 300K-parameter model roughly matching a 2.14M-parameter baseline on quark-gluon tagging, though these are single-benchmark claims from a preprint that has not been peer reviewed. For ML readers, the useful takeaway is the sparsity-by-momentum design pattern, which may transfer to other structured, variable-length inputs.
This is SParTA: A modular, efficient and scalable transformer architecture for jet-tagging
arXiv:2610.27106v1 Announce Type: cross Abstract: We present the Sparse Particle-interaction Transformer Architecture (SParTA), a modular, scalable, and efficient transformer for jet tagging at high-energy colliders. Exploiting the steeply falling transverse-momentum ($p_T$) spectrum of jet constituents, SParTA factorizes constituent processing into two topological regimes. Ordered by descending $p_T$, only the $N$ hardest constituents (the hard core) participate in quadratic $O(N^2)$ differential interaction attention driven by pairwise physical and learned relational features. The remaining soft constituents are integrated via linear $O(lN)$ cross-attention to the hard core, preserving soft-radiation and color-flow information while avoiding the $O(l^2)$ memory bottleneck of full self-attention over all $l$ constituents. Pairwise interaction features are computed dynamically in-network from constituent four-momenta and conditioned on global jet-level substructure observables. On the quark-gluon tagging benchmark, a 300K-parameter SParTA model achieves an AUC of 92.02% and accuracy of 85.06%, matching the 2.14M-parameter Particle Transformer (ParT) at one-seventh the parameter count and outperforming IAFormer by 21% in background rejection at 30% signal efficiency. On the boosted top-tagging benchmark, SParTA outperforms ParT and ParticleNet with 511K parameters, while we analyze the architectural trade-offs relative to fully dense multi-layer interaction networks.
Abstract:We present the Sparse Particle-interaction Transformer Architecture (SParTA), a modular, scalable, and efficient transformer for jet tagging at high-energy colliders. Exploiting the steeply falling transverse-momentum ($p_T$) spectrum of jet constituents, SParTA factorizes constituent processing into two topological regimes. Ordered by descending $p_T$, only the $N$ hardest constituents (the hard core) participate in quadratic $O(N^2)$ differential interaction attention driven by pairwise physical and learned relational features. The remaining soft constituents are integrated via linear $O(lN)$ cross-attention to the hard core, preserving soft-radiation and color-flow information while avoiding the $O(l^2)$ memory bottleneck of full self-attention over all $l$ constituents. Pairwise interaction features are computed dynamically in-network from constituent four-momenta and conditioned on global jet-level substructure observables. On the quark-gluon tagging benchmark, a 300K-parameter SParTA model achieves an AUC of 92.02% and accuracy of 85.06%, matching the 2.14M-parameter Particle Transformer (ParT) at one-seventh the parameter count and outperforming IAFormer by 21% in background rejection at 30% signal efficiency. On the boosted top-tagging benchmark, SParTA outperforms ParT and ParticleNet with 511K parameters, while we analyze the architectural trade-offs relative to fully dense multi-layer interaction networks.
High Energy Physics - Phenomenology (hep-ph); Machine Learning (cs.LG); High Energy Physics - Experiment (hep-ex)
arXiv:2610.27106 [hep-ph]
(or arXiv:2610.27106v1 [hep-ph] for this version)
https://doi.org/10.48550/arXiv.2610.27106 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 287 words
Related coverage
Optimizing Transformer Neural Network for Real-Time Outlier Detection on FPGAs
arXiv · Jul 28, 2026
Physics Transformer: Tailoring Transformer for General PDE Prediction
arXiv · Jul 28, 2026
Plain Transformers Can be Powerful Graph Learners
arXiv · Aug 28, 2026
Latent reasoning architectures would likely undermine CoT, our strongest oversight tool
AI Alignment Forum · Sep 23, 2026