aiqbench analysis
This research addresses the computational intractability of neural ensemble search by introducing a dual-objective surrogate model that simultaneously estimates predictive accuracy and diversity. By representing candidate architectures as directed acyclic graphs, the method efficiently identifies high-performing ensembles without exhaustive exploration of the joint optimization space. The approach demonstrates competitive results on standard benchmarks like CIFAR-10 and FashionMNIST compared to traditional baselines such as Deep Ensembles.
Surrogate assisted diversity estimation in neural ensemble search
arXiv:2607.26940v1 Announce Type: new Abstract: Ensembles are a standard way to improve the performance and robustness of deep neural networks, but their effectiveness crucially depends on both the quality and the diversity of individual models. Most neural architecture search (NAS) methods are computationally expensive. Extending them to neural ensemble search (NES), which requires joint optimization of individual architectures and their ensemble composition, leads to an exponential growth of the search space and makes the problem computationally intractable. To address this, we introduce a dual-objective surrogate-guided ensemble search: candidate architectures are represented as directed acyclic graphs, and two surrogate models are trained independently to estimate predictive accuracy and diversity potential. Their combined estimates guide an NES framework that efficiently identifies architectures that are both individually strong and collectively diverse. Our final ensemble achieves competitive or superior performance compared to standard baselines such as Deep Ensembles and Random Search on FashionMNIST, CIFAR-10, and CIFAR-100.
Abstract:Ensembles are a standard way to improve the performance and robustness of deep neural networks, but their effectiveness crucially depends on both the quality and the diversity of individual models. Most neural architecture search (NAS) methods are computationally expensive. Extending them to neural ensemble search (NES), which requires joint optimization of individual architectures and their ensemble composition, leads to an exponential growth of the search space and makes the problem computationally intractable. To address this, we introduce a dual-objective surrogate-guided ensemble search: candidate architectures are represented as directed acyclic graphs, and two surrogate models are trained independently to estimate predictive accuracy and diversity potential. Their combined estimates guide an NES framework that efficiently identifies architectures that are both individually strong and collectively diverse. Our final ensemble achieves competitive or superior performance compared to standard baselines such as Deep Ensembles and Random Search on FashionMNIST, CIFAR-10, and CIFAR-100.
The paper was presented at the "Artificial Intelligence Applications and Innovations 2026" conference. The final publication is available at this https URL
Machine Learning (cs.LG)
arXiv:2607.26940 [cs.LG]
(or arXiv:2607.26940v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2607.26940 arXiv-issued DOI via DataCite (pending registration)
IFIP Advances in Information and Communication Technology, vol 792. Springer, Cham. 2027
https://doi.org/10.1007/978-3-032-30612-8_12 DOI(s) linking to related resources
Full text under the publisher's open licence · 205 words
Related coverage
Surrogate-Assisted Genetic Programming with Phenotypic Characterisation in Dynamic Multi-Mode Project Scheduling
arXiv · Sep 16, 2026
NeuroPareto: Calibrated Acquisition for Costly Many-Goal Search in Vast Parameter Spaces
arXiv · Jul 23, 2026
DreamTest: World-Model Surrogates for Search-Based Testing of Deep Reinforcement Learning Agents
arXiv · Oct 6, 2026
The J-Space Debate, Agent Swarms, and Pacing Frontier AI - Digital Minds Newsletter #4
LessWrong · Sep 18, 2026