analiza aiqbench
This research introduces jBOT, a self-supervised pre-training method using self-distillation on LHC jet data that successfully learns semantic representations without labels. The approach demonstrates emergent clustering in the embedding space, enabling effective anomaly detection and improved classification performance for high-energy physics tasks. By bridging advanced machine learning techniques with particle physics data analysis, this work advances the application of foundation models in scientific discovery.
jBOT: Semantic Jet Representation Clustering Emerges from Self-Distillation
arXiv:2601.11719v4 Announce Type: replace Abstract: Self-supervised learning, in the context of foundation model training, is a powerful pre-training method for learning feature representations without labels, which often capture generic underlying semantics from the data and can later be fine-tuned for downstream tasks. In this work, we introduce jBOT, a pre-training method based on self-distillation for jet data from the CERN Large Hadron Collider, which combines local particle-level distillation with global jet-level distillation to learn jet representations that support downstream tasks such as anomaly detection and classification. We observe that pre-training on unlabeled jets leads to emergent semantic class clustering in the representation space. The clustering in the frozen embedding, when pre-trained on background jets only, enables anomaly detection via simple distance-based metrics, and the learned embedding can be fine-tuned for classification with improved performance compared to supervised models trained from scratch.
Abstract:Self-supervised learning, in the context of foundation model training, is a powerful pre-training method for learning feature representations without labels, which often capture generic underlying semantics from the data and can later be fine-tuned for downstream tasks. In this work, we introduce jBOT, a pre-training method based on self-distillation for jet data from the CERN Large Hadron Collider, which combines local particle-level distillation with global jet-level distillation to learn jet representations that support downstream tasks such as anomaly detection and classification. We observe that pre-training on unlabeled jets leads to emergent semantic class clustering in the representation space. The clustering in the frozen embedding, when pre-trained on background jets only, enables anomaly detection via simple distance-based metrics, and the learned embedding can be fine-tuned for classification with improved performance compared to supervised models trained from scratch.
Machine Learning (cs.LG); High Energy Physics - Experiment (hep-ex)
arXiv:2601.11719 [cs.LG]
(or arXiv:2601.11719v4 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2601.11719 Focus to learn more arXiv-issued DOI via DataCite
SciPost Phys. 21, 053 (2026)
https://doi.org/10.21468/SciPostPhys.21.3.053 Focus to learn more DOI(s) linking to related resources
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 249 cuvinte
Subiecte conexe
Semi-Supervised Text-Attributed Graph Distillation
arXiv · 24 iul. 2026
Exploring Hierarchical Interest Representation For Meta Ads Deep Funnel Optimization
Meta Engineering · 15 iul. 2026
Thousand-dimensional structure
AI Alignment Forum · 30 iul. 2026
PIRAMID: Progress and Plans
LessWrong · 27 iul. 2026