aiqbench analysis
This research introduces a scalable Bayesian nonparametric learner that overcomes the conjugacy limitations of standard Stochastic Variational Inference by utilizing adaptive step-sizes inspired by Adam. The method enables efficient large-scale training on complex, non-conjugate models without requiring closed-form variational posterior expectations, demonstrating compatibility with deep features like ResNet for high-class-count datasets. By improving convergence in stochastic settings through constant and adaptive stepsizes, the approach offers a significant computational advantage over Monte Carlo-based alternatives while maintaining or exceeding state-of-the-art clustering performance.
PYPM-GGD: Pitman-Yor Process Mixture with Generalized Gaussian Density using ADAM
arXiv:2607.24583v1 Announce Type: new Abstract: Large scale Bayesian nonparametrics (BNP) learner such as Stochastic Variational Inference (SVI) can handle datasets with large class number and large training size at fractional cost. Like its predecessor, SVI rely on the assumption of conjugate variational posterior to approximate the true posterior. A more challenging problem is to consider large scale learning on non-conjugate posterior. Recent works in this direction are mostly associated with using Monte Carlo methods for approximating the learner. However, these works are usually demonstrated on non-BNP related task and less complex models such as logistic regression, due to higher computational complexity. In order to overcome the issue faced by SVI, we develop a novel approach based on the recently proposed constant stepsize stochastic gradient ascent to allow large scale learning on non-conjugate posterior. Unlike SVI, our new learner does not require closed- form expression for the variational posterior expectatations. Our only requirement is that the variational posterior is differentiable. In order to ensure convergence in stochastic settings, SVI rely on decaying step-sizes to slow its learning. Inspired by SVI and Adam, we propose the novel use of adaptive stepsizes in our method to significantly improve its learning. We show that our proposed methods is compatible with ResNet features when applied to large class number datasets such as MIT67 and SUN397. Finally, we compare our proposed learner with several recent works such as deep clustering algorithms and showed we were able to produce on-par or outperform the state-of-the-art methods in terms of clustering measures.
Abstract:Large scale Bayesian nonparametrics (BNP) learner such as Stochastic Variational Inference (SVI) can handle datasets with large class number and large training size at fractional cost. Like its predecessor, SVI rely on the assumption of conjugate variational posterior to approximate the true posterior. A more challenging problem is to consider large scale learning on non-conjugate posterior. Recent works in this direction are mostly associated with using Monte Carlo methods for approximating the learner. However, these works are usually demonstrated on non-BNP related task and less complex models such as logistic regression, due to higher computational complexity. In order to overcome the issue faced by SVI, we develop a novel approach based on the recently proposed constant stepsize stochastic gradient ascent to allow large scale learning on non-conjugate posterior. Unlike SVI, our new learner does not require closed- form expression for the variational posterior expectatations. Our only requirement is that the variational posterior is differentiable. In order to ensure convergence in stochastic settings, SVI rely on decaying step-sizes to slow its learning. Inspired by SVI and Adam, we propose the novel use of adaptive stepsizes in our method to significantly improve its learning. We show that our proposed methods is compatible with ResNet features when applied to large class number datasets such as MIT67 and SUN397. Finally, we compare our proposed learner with several recent works such as deep clustering algorithms and showed we were able to produce on-par or outperform the state-of-the-art methods in terms of clustering measures.
Machine Learning (cs.LG)
arXiv:2607.24583 [cs.LG]
(or arXiv:2607.24583v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2607.24583 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 342 words
Related coverage
Regular Fourier Features for Nonstationary Gaussian Processes
arXiv · Sep 24, 2026
Guidance for Prior Change via Density Ratio Estimation
arXiv · Aug 25, 2026
Physics-informed random neural architectures on compact boundaryless multiply connected manifolds for random-operator models in machine learning and uncertainty quantification
arXiv · Oct 14, 2026
Projected Wasserstein-Fisher-Rao Flows for Hawkes Background-Intensity Estimation
arXiv · Oct 15, 2026