aiqbench analysis
This study demonstrates how self-supervised learning via masked autoencoders can effectively leverage vast amounts of unlabeled auroral spectral data to outperform traditional supervised baselines. By pretraining a 1D Vision Transformer on unlabelled spectra, the model successfully recovers key physical emission-line ratios and achieves superior classification performance with significantly fewer expert annotations. The analysis also highlights the limitations of cross-domain transfer, showing that foundation models trained in different spectral windows (e.g., infrared) fail to generalize to optical auroral data without domain-specific pretraining.
Self-Supervised Representation Learning: From Spectral Foundation Models to Auroral Emission Spectra
arXiv:2609.31206v1 Announce Type: new Abstract: Auroral spectrographs such as the Auroral Spectrograph In Skibotn (ASIS) record hundreds of thousands of emission spectra, but only a few hundred can be labelled by an expert. To exploit the rest, we pretrain a 1D Vision Transformer with a masked autoencoder on 223,000 unlabelled spectra. Without labels, its representation recovers the emission-line intensity ratios that physicists use to diagnose the precipitating particles (R^2 0.91 vs. 0.77 for an untrained control) and, under one linear probe, classifies as well as 13 features designed by experts. Fine-tuned, the model outperforms the previous supervised auroral classifier on its own benchmark (macro-AP 88.5 vs. 77.8), reaches 0.870 mAP, and exceeds the same architecture trained from scratch by +0.159 with 10% of the labels; attribution shows that it uses both N2+ bands. Could an existing pretrained model replace it? Two astronomical spectral foundation models and a time-series model transfer according to their spectral window: SpectraFM, trained in the infrared, falls below the untrained control, whereas SpecFormer, trained in the optical, approaches in-domain pretraining without reaching it.
Abstract:Auroral spectrographs such as the Auroral Spectrograph In Skibotn (ASIS) record hundreds of thousands of emission spectra, but only a few hundred can be labelled by an expert. To exploit the rest, we pretrain a 1D Vision Transformer with a masked autoencoder on 223,000 unlabelled spectra. Without labels, its representation recovers the emission-line intensity ratios that physicists use to diagnose the precipitating particles (R^2 0.91 vs. 0.77 for an untrained control) and, under one linear probe, classifies as well as 13 features designed by experts. Fine-tuned, the model outperforms the previous supervised auroral classifier on its own benchmark (macro-AP 88.5 vs. 77.8), reaches 0.870 mAP, and exceeds the same architecture trained from scratch by +0.159 with 10% of the labels; attribution shows that it uses both N2+ bands. Could an existing pretrained model replace it? Two astronomical spectral foundation models and a time-series model transfer according to their spectral window: SpectraFM, trained in the infrared, falls below the untrained control, whereas SpecFormer, trained in the optical, approaches in-domain pretraining without reaching it.
5 pages, 1 figure, 3 tables. Submitted to IEEE ICASSP 2027
Machine Learning (cs.LG); Space Physics (physics.space-ph)
arXiv:2609.31206 [cs.LG]
(or arXiv:2609.31206v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2609.31206 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 280 words
Related coverage
Flattening the Connectome Spectrum: A Spectral Filter for FC Induces a Pretraining Target for fMRI Encoders
arXiv · Sep 30, 2026
BrainNext: A General-Purpose Self-Supervised Foundation Model for Brain MRI Analysis
arXiv · Jul 21, 2026
Toward a foundation model for forest point clouds
arXiv · Sep 22, 2026
Lightweight Semantic EEG Foundation Model for Frozen Cross-Disorder Transfer
arXiv · Oct 6, 2026