aiqbench analysis
This research introduces a hierarchical Bayesian multitask learning framework designed to improve disease prediction by leveraging shared sparsity structures across heterogeneous microbiome datasets. By utilizing variational inference, the method achieves robust performance and reliable uncertainty quantification even when pooling data from diverse experimental setups and patient demographics. The approach advances machine learning methodologies for handling complex, multi-task biological classification problems where traditional models struggle with data heterogeneity.
Hierarchical Sparse Bayesian Multitask Learning for Disease Prediction in Pooled Microbiome Studies
arXiv:2502.02552v2 Announce Type: replace Abstract: This paper proposes a hierarchical Bayesian multitask learning model that is applicable to the general multi-task binary classification learning problem where the model assumes a shared sparsity structure across different tasks. We derive a computationally efficient inference algorithm based on variational inference to approximate the posterior distribution. We demonstrate the potential of the new approach on various synthetic datasets and for predicting human health status based on microbiome profile. Our analysis incorporates data pooled from multiple microbiome studies, along with a comprehensive comparison with other benchmark methods. Results in synthetic datasets show that the proposed approach has superior support recovery property when the underlying regression coefficients share a common sparsity structure across different tasks. Our experiments on microbiome classification demonstrate the utility of the method in extracting informative taxa while providing well-calibrated predictions with uncertainty quantification and achieving competitive performance in terms of prediction metrics. Notably, despite the heterogeneity of the pooled datasets (e.g., different experimental objectives, laboratory setups, sequencing equipment, patient demographics), our method delivers robust results.
Authors:Haonan Zhu, Andre R. Goncalves, Camilo Valdes, Hiranmayi Ranganathan, Boya Zhang, Jose Manuel Martí, Car Reen Kok, Monica K. Borucki, Nisha J. Mulakken, James B. Thissen, Crystal Jaing, Alfred Hero, Nicholas A. Be
Abstract:This paper proposes a hierarchical Bayesian multitask learning model that is applicable to the general multi-task binary classification learning problem where the model assumes a shared sparsity structure across different tasks. We derive a computationally efficient inference algorithm based on variational inference to approximate the posterior distribution. We demonstrate the potential of the new approach on various synthetic datasets and for predicting human health status based on microbiome profile. Our analysis incorporates data pooled from multiple microbiome studies, along with a comprehensive comparison with other benchmark methods. Results in synthetic datasets show that the proposed approach has superior support recovery property when the underlying regression coefficients share a common sparsity structure across different tasks. Our experiments on microbiome classification demonstrate the utility of the method in extracting informative taxa while providing well-calibrated predictions with uncertainty quantification and achieving competitive performance in terms of prediction metrics. Notably, despite the heterogeneity of the pooled datasets (e.g., different experimental objectives, laboratory setups, sequencing equipment, patient demographics), our method delivers robust results.
Machine Learning (cs.LG); Biomolecules (q-bio.BM); Applications (stat.AP); Computation (stat.CO); Methodology (stat.ME)
arXiv:2502.02552 [cs.LG]
(or arXiv:2502.02552v2 [cs.LG] for this version)
Zhu, H., Goncalves, A.R., Kok, C.R. et al. Hierarchical sparse Bayesian multitask learning for disease prediction in pooled microbiome studies. BioData Mining (2026)
https://doi.org/10.1186/s13040-026-00600-x DOI(s) linking to related resources
Full text under the publisher's open licence · 249 words
Related coverage
PIRAMID: Progress and Plans
LessWrong · Jul 27, 2026
Constrained Bayesian Optimization for Hierarchical Federated Learning in IoT Networks for Plant Disease Classification
arXiv · Sep 9, 2026
DTVEM-RE: A Hierarchical Random-Effects Extension of the Differential Time-Varying Effect Model for Person-Specific Multi-Lag Estimation in Intensive Longitudinal Data
arXiv · Jun 15, 2026
Generalization and infinite width
LessWrong · Aug 1, 2026