analiza aiqbench
This paper is a theoretical and numerical study of the frequency principle, the observation that neural networks trained by gradient descent tend to fit low-frequency components of a target before high-frequency ones. It proves this under assumptions for finite linear approximation classes, and shows that low-frequency priority does not guarantee that low frequencies always converge faster, since for some frequency pairs the higher one is the slower to learn. For ML readers, the useful takeaway is a more precise, bounded understanding of spectral bias in simple linear models, with the caveat that the results are validated on Legendre and trigonometric bases rather than deep networks.
General Frequency Principle for Linear Approximation Classes
arXiv:2610.41179v1 Announce Type: new Abstract: In this paper, we study a general frequency principle under gradient descent training for the finite linear approximation classes. Under suitable assumptions, we prove that the learner amplitude relative to the target amplitude vanishes at high frequencies, uniformly throughout training. We also establish sufficient conditions for low-frequency priority, covering single-frequency targets trained separately and two frequency components trained in the same run. This general frequency principle does not imply that lower frequencies always converge faster. Instead, for two frequencies that satisfy certain comparison conditions, one component will take longer to learn than the other; and in common cases, the longer component has higher frequency. Numerical experiments with Legendre and trigonometric bases validate the theoretical results.
Abstract:In this paper, we study a general frequency principle under gradient descent training for the finite linear approximation classes. Under suitable assumptions, we prove that the learner amplitude relative to the target amplitude vanishes at high frequencies, uniformly throughout training. We also establish sufficient conditions for low-frequency priority, covering single-frequency targets trained separately and two frequency components trained in the same run. This general frequency principle does not imply that lower frequencies always converge faster. Instead, for two frequencies that satisfy certain comparison conditions, one component will take longer to learn than the other; and in common cases, the longer component has higher frequency. Numerical experiments with Legendre and trigonometric bases validate the theoretical results.
Machine Learning (cs.LG); Numerical Analysis (math.NA)
arXiv:2610.41179 [cs.LG]
(or arXiv:2610.41179v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.41179 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 212 cuvinte
Subiecte conexe
Tulane Researchers Boost Superoscillation Frequency With Two Waves
Quantum Zeitgeist · 6 aug. 2026
Synthesizing superoscillations with just two frequencies
arXiv · 23 iul. 2026
SAGE: Source-Anchored Guidance via Frequency Equalization for Hierarchical RGB-T Alignment and Fusion
arXiv · 28 sept. 2026
Rotating-wave approximation for spin-boson models with structured fields
arXiv · 1 sept. 2026