analiza aiqbench
This is a theoretical arXiv preprint (v1, not yet peer reviewed) on how efficiently different architectures can represent functions. It argues that universal approximation alone doesn't separate models, and that depth and higher-order interactions expand which functions a model can represent efficiently. For ML practitioners, the useful takeaway is a formal basis for why deep Boltzmann machines may be more expressive per parameter than restricted ones, though the results concern representation rather than learnability or practical training performance.
Going deep, reaching further: What can Boltzmann machines efficiently represent?
arXiv:2610.39731v1 Announce Type: new Abstract: While universal approximation theorems theoretically underpin the representation capabilities of machine learning models, they have two shortcomings: They do not focus on efficient representations and they tend to be insensitive to depth and other heuristically important properties. In this work we show that these two shortcomings are intertwined. We prove that machine learning models which are deeper or have higher-order interactions can efficiently represent a larger set of functions, i.e. have larger efficient reach. It follows that deep Boltzmann machines (DBMs) have maximal efficient reach, restricted Boltzmann machines (RBMs) sub-maximal, and k-body generalizations of RBMs form a hierarchy between the two, even though all are universal. More generally, (i) we provide a full characterization of machine learning models with maximal efficient reach; (ii) we derive separations between deep and shallow models for three representation tasks; and (iii) we prove that the efficient reach of shallow models constitutes an infinite hierarchy indexed by the interaction order. The proofs draw on different methodologies, namely on (i) a recent characterization of universal spin models; (ii) computational complexity bounds for spin models; and (iii) communication complexity bounds applied to shallow models. This work shows that universal machine learning models can strongly differ in their efficient representation capabilities, and provides a theoretical foundation for heuristic observations.
Title:Going deep, reaching further: What can Boltzmann machines efficiently represent?
Abstract:While universal approximation theorems theoretically underpin the representation capabilities of machine learning models, they have two shortcomings: They do not focus on efficient representations and they tend to be insensitive to depth and other heuristically important properties. In this work we show that these two shortcomings are intertwined. We prove that machine learning models which are deeper or have higher-order interactions can efficiently represent a larger set of functions, i.e. have larger efficient reach. It follows that deep Boltzmann machines (DBMs) have maximal efficient reach, restricted Boltzmann machines (RBMs) sub-maximal, and k-body generalizations of RBMs form a hierarchy between the two, even though all are universal. More generally, (i) we provide a full characterization of machine learning models with maximal efficient reach; (ii) we derive separations between deep and shallow models for three representation tasks; and (iii) we prove that the efficient reach of shallow models constitutes an infinite hierarchy indexed by the interaction order. The proofs draw on different methodologies, namely on (i) a recent characterization of universal spin models; (ii) computational complexity bounds for spin models; and (iii) communication complexity bounds applied to shallow models. This work shows that universal machine learning models can strongly differ in their efficient representation capabilities, and provides a theoretical foundation for heuristic observations.
Machine Learning (cs.LG); Disordered Systems and Neural Networks (cond-mat.dis-nn); Mathematical Physics (math-ph)
arXiv:2610.39731 [cs.LG]
(or arXiv:2610.39731v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.39731 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 322 cuvinte
Subiecte conexe
Training Energy-Based Models with Non-MCMC Samplers and Efficient Temperature Estimation
arXiv · 16 sept. 2026
FUND: Density Flow for Sampling Unnormalised Distributions
arXiv · 30 sept. 2026
Scalable Inference-Time Annealing with Surrogate Likelihood Estimators
arXiv · 9 iun. 2026
Boltzmann transport theory of magnon-exciton drag
arXiv · 10 iun. 2026