aiqbench analysis
This research proposes structured hyperparameter sharing within Gaussian processes to address overfitting in high-dimensional image data, offering a parameter-efficient alternative to fully anisotropic kernels. By tying lengthscales across adjacent pixels and color channels, the method improves predictive performance on benchmarks like MNIST and satellite imagery while enabling non-local generalization.
Structured Relevance Determination and Channel Additive Gaussian Processes for Image Data
arXiv:2610.25385v1 Announce Type: new Abstract: We introduce radial basis function (RBF) kernels for Gaussian processes that use structured hyperparameter sharing for image-valued inputs. For high-dimensional inputs such as images, a fully anisotropic RBF kernel assigns a separate lengthscale to each input dimension, which can introduce many hyperparameters and may lead to overfitting and poorly calibrated predictive uncertainty when data are limited. We propose structured relevance determination by tying lengthscales across groups of adjacent pixels and within each colour channel. This yields patch-wise and channel-wise constrained anisotropic RBF parametrisations that reduce the number of independently learned lengthscales. We further introduce channel-additive Gaussian processes for multi-channel images and combine them with structured relevance determination, resulting in a parameter-efficient decomposition of the latent function into components defined on channel subsets, enabling some non-local generalisation. The proposed kernels are evaluated on MNIST and a compact binary aircraft-detection dataset of low-resolution RGB satellite image chips. We also show how channel-wise structured relevance determination can be incorporated into convolutional Gaussian processes. We observe improved predictive performance compared to isotropic and fully anisotropic RBF baselines.
Abstract:We introduce radial basis function (RBF) kernels for Gaussian processes that use structured hyperparameter sharing for image-valued inputs. For high-dimensional inputs such as images, a fully anisotropic RBF kernel assigns a separate lengthscale to each input dimension, which can introduce many hyperparameters and may lead to overfitting and poorly calibrated predictive uncertainty when data are limited. We propose structured relevance determination by tying lengthscales across groups of adjacent pixels and within each colour channel. This yields patch-wise and channel-wise constrained anisotropic RBF parametrisations that reduce the number of independently learned lengthscales. We further introduce channel-additive Gaussian processes for multi-channel images and combine them with structured relevance determination, resulting in a parameter-efficient decomposition of the latent function into components defined on channel subsets, enabling some non-local generalisation. The proposed kernels are evaluated on MNIST and a compact binary aircraft-detection dataset of low-resolution RGB satellite image chips. We also show how channel-wise structured relevance determination can be incorporated into convolutional Gaussian processes. We observe improved predictive performance compared to isotropic and fully anisotropic RBF baselines.
Machine Learning (cs.LG)
arXiv:2610.25385 [cs.LG]
(or arXiv:2610.25385v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.25385 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 265 words
Related coverage
PXRD-Bench: How Close Are We to Determining Crystal Structures from Powder Diffraction?
arXiv · Oct 15, 2026
Fold'EM: Direct atomic structure inference from Cryo-EM particles
arXiv · Oct 2, 2026
EDGE: Engine for Deterministic Graph Evaluation through Conversation Simulation from Graph Structured DSL Configuration
arXiv · Sep 1, 2026
MolLangData: A Large-Scale Dataset for Molecular Structure-Language Description via a Rule-Regularized Method
arXiv · Sep 30, 2026