analiza aiqbench
This research demonstrates how sharing parameters across iterative loops in neural networks can achieve superior statistical accuracy under strict memory budgets compared to conventional untied models. By establishing rigorous bounds on risk for various likelihood models, the study proves that repeated computation with fixed parameters can effectively approximate complex functions without increasing model size. This theoretical advance offers a viable pathway for deploying high-accuracy AI systems in resource-constrained environments where parameter efficiency is critical.
Statistical Gains from Looped Estimation under Parameter Budgets
arXiv:2609.25778v1 Announce Type: cross Abstract: Growing memory demands in artificial intelligence motivate learning with fewer trainable parameters. We ask whether a looped estimator, which repeatedly applies one fitted operator with parameters shared across iterations, can improve statistical accuracy under a common parameter budget. Its conventional untied counterpart uses separate parameters at each iteration. For general likelihood models, we establish an upper bound on squared Hellinger risk for looped sieve maximum likelihood and a minimax lower bound over the tuned untied family. These bounds reveal a parameter--iteration--accuracy tradeoff: repeated computation can improve approximation without adding parameters, while increasing computational cost and fitted-class complexity. For targets of known H\"older smoothness, looped residual feedforward networks and a specified post-layer-normalized Transformer attain the minimax polynomial rate up to logarithmic factors with a fixed number of bounded real parameters. At sufficiently large fixed budgets, looped worst-case risk vanishes as sample size grows, whereas optimal worst-case untied risk remains bounded away from zero. Under specified growing-budget conditions, the loop-to-untied risk ratio also tends to zero. Gaussian and Laplace regression, binary response, and energy-based density estimation illustrate the theory.
Abstract:Growing memory demands in artificial intelligence motivate learning with fewer trainable parameters. We ask whether a looped estimator, which repeatedly applies one fitted operator with parameters shared across iterations, can improve statistical accuracy under a common parameter budget. Its conventional untied counterpart uses separate parameters at each iteration. For general likelihood models, we establish an upper bound on squared Hellinger risk for looped sieve maximum likelihood and a minimax lower bound over the tuned untied family. These bounds reveal a parameter--iteration--accuracy tradeoff: repeated computation can improve approximation without adding parameters, while increasing computational cost and fitted-class complexity. For targets of known Hölder smoothness, looped residual feedforward networks and a specified post-layer-normalized Transformer attain the minimax polynomial rate up to logarithmic factors with a fixed number of bounded real parameters. At sufficiently large fixed budgets, looped worst-case risk vanishes as sample size grows, whereas optimal worst-case untied risk remains bounded away from zero. Under specified growing-budget conditions, the loop-to-untied risk ratio also tends to zero. Gaussian and Laplace regression, binary response, and energy-based density estimation illustrate the theory.
Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST)
arXiv:2609.25778 [stat.ML]
(or arXiv:2609.25778v1 [stat.ML] for this version)
https://doi.org/10.48550/arXiv.2609.25778 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 277 cuvinte
Subiecte conexe
When and Why LLM Causal Priors Help: Closed-Loop Prior Selection for Amortized Causal Inference
arXiv · 9 sept. 2026
Expenditure Horizon: Measuring Optimization Ability, with an Application to NanoGPT
METR · 21 iul. 2026
[AINews] not much happened today
Latent Space · 3 oct. 2026
[AINews] DeepSeek v4.1-Flash: 763B-P8B-D16B novel causal Encoder–Decoder architecture with vision marks the Return of the Whale
Latent Space · 12 sept. 2026