aiqbench analysis
This study evaluates probabilistic machine learning techniques, specifically Monte Carlo dropout on convolutional neural networks, for classifying astronomical spectra with calibrated uncertainty estimates. By balancing high classification accuracy with efficient inference and reliable error quantification, the research offers a practical framework for handling data uncertainties in large-scale surveys like 4MOST. The findings highlight how Bayesian methods can enhance standard deep learning models without significant computational overhead, directly addressing reliability needs in AI-driven scientific discovery.
Bayesian classification of astronomical spectra with class uncertainties
arXiv:2609.21694v1 Announce Type: cross Abstract: Context: We developed a probabilistic machine learning method with the aim of performing the O(10)-way classification of low- and high-resolution spectra of stellar and extragalactic targets for the upcoming 4MOST survey. In fulfilment of the survey requirements, this method should be able to express uncertainty in the input data as well as uncertainty introduced in its prediction. Aims: Four different methods are explored: (1) convolutional neural networks (CNNs), (2) the Dirichlet distribution, (3) Monte Carlo dropout (MCD), (4) Bayesian neural Networks (BNNs) + variational inference (VI). Training and validation was performed using labelled spectra from the SDSS database and a custom 4MOST mock dataset. All the methods were compared in terms of the same metrics: accuracy, area under the curve (AUC), expected calibration error (ECE), Shannon entropy, negative log-likelihood (NLL), Brier score, training time, and inference time. Methods: A CNN with simple architecture and about 20,000 parameters was trained to achieve classification accuracies of 91.5% on SDSS data and 92.8% on 4MOST mock data. The direct Dirichlet prediction and VI models tested provide uncertainties on class membership probabilities, but they confuse classes more often. The MCD on a CNN is found to be the most suitable; it boosts the point-estimate accuracies to 92.6% and 93.9%, while still providing fast training and sufficiently fast inference. Compared to a standard CNN, the method additionally provides well-calibrated uncertainties at marginal extra cost.
Abstract:Context: We developed a probabilistic machine learning method with the aim of performing the O(10)-way classification of low- and high-resolution spectra of stellar and extragalactic targets for the upcoming 4MOST survey. In fulfilment of the survey requirements, this method should be able to express uncertainty in the input data as well as uncertainty introduced in its prediction. Aims: Four different methods are explored: (1) convolutional neural networks (CNNs), (2) the Dirichlet distribution, (3) Monte Carlo dropout (MCD), (4) Bayesian neural Networks (BNNs) + variational inference (VI). Training and validation was performed using labelled spectra from the SDSS database and a custom 4MOST mock dataset. All the methods were compared in terms of the same metrics: accuracy, area under the curve (AUC), expected calibration error (ECE), Shannon entropy, negative log-likelihood (NLL), Brier score, training time, and inference time. Methods: A CNN with simple architecture and about 20,000 parameters was trained to achieve classification accuracies of 91.5% on SDSS data and 92.8% on 4MOST mock data. The direct Dirichlet prediction and VI models tested provide uncertainties on class membership probabilities, but they confuse classes more often. The MCD on a CNN is found to be the most suitable; it boosts the point-estimate accuracies to 92.6% and 93.9%, while still providing fast training and sufficiently fast inference. Compared to a standard CNN, the method additionally provides well-calibrated uncertainties at marginal extra cost.
Instrumentation and Methods for Astrophysics (astro-ph.IM); Machine Learning (cs.LG); Data Analysis, Statistics and Probability (physics.data-an)
arXiv:2609.21694 [astro-ph.IM]
(or arXiv:2609.21694v1 [astro-ph.IM] for this version)
https://doi.org/10.48550/arXiv.2609.21694 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
https://doi.org/10.1051/0004-6361/202556350 Focus to learn more DOI(s) linking to related resources
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 345 words
Related coverage
The Entangled Dimensions of Decision Theory
LessWrong · Jul 30, 2026
An Uncertainty-Driven Hybrid Deep Learning Approach for Broad-Coverage RF Modulation Recognition
arXiv · Aug 4, 2026
Dynamic Decision-Making under Model Misspecification: A Stochastic Stability Approach
arXiv · Jul 21, 2026
Uncertainty-Aware Calibrated Clinical Text Classification with Large Language Models
arXiv · Sep 16, 2026