analiza aiqbench
This paper establishes a rigorous mathematical equivalence between neural network training and the search through viscous Hamilton-Jacobi initial-value problems, unifying perspectives from tropical algebra to convex optimization. By framing backpropagation as a co-state equation within a Hamiltonian system, it provides exact structural insights for architectures ranging from residual networks to transformers that vanish in specific limits. The work offers quantified error bounds and implications for generalization rates and adversarial robustness grounded in dynamical systems theory rather than empirical observation alone.
The Hamilton-Jacobi Theory of Deep Learning
arXiv:2605.28983v2 Announce Type: replace Abstract: In this paper, training a neural network is identified, exactly, as a search through Hamilton--Jacobi initial-value problems: each gradient step selects the initial data of a viscous Hamilton--Jacobi equation whose Hopf--Cole propagator best fits the observations; at inference, the input is the spatial point at which that solution is evaluated and the initial condition is already encoded in the weights. The correspondence is exact for log-sum-exp layers, with ReLU, sigmoid, SiLU, and GELU each an exact limit, gradient, or moment of the same object, and exact in composition across depth and width, with a quantified error at finite depth that vanishes in the joint limit. It is structural for residual networks, transformers, and recurrent networks (RNNs, LSTMs, SSMs), each discretizing the same class of equations, at a named and quantified approximation error. A single deformation parameter $\varepsilon$ unifies all four perspectives (network, tropical algebra, viscous PDE, convex optimization) in a commutative diagram closed under Lipschitz conditions. Quantitative consequences include: the minimax optimal generalization rate $O(n^{-1/(d+2)})$ for fixed $t$; adversarial robustness controlled by $\varepsilon$; backpropagation as the co-state equation of the Hamiltonian system for residual networks (Pontryagin Maximum Principle); scaling exponents consistent with data intrinsic dimension via PDE quadrature; and a closed-form $O(N)$ influence function (softmax attribution weights $\pi_j$) whose entropy landscape undergoes fold bifurcations as $\varepsilon$ increases, each merging attribution basins.
Abstract:In this paper, training a neural network is identified, exactly, as a search through Hamilton--Jacobi initial-value problems: each gradient step selects the initial data of a viscous Hamilton--Jacobi equation whose Hopf--Cole propagator best fits the observations; at inference, the input is the spatial point at which that solution is evaluated and the initial condition is already encoded in the weights. The correspondence is exact for log-sum-exp layers, with ReLU, sigmoid, SiLU, and GELU each an exact limit, gradient, or moment of the same object, and exact in composition across depth and width, with a quantified error at finite depth that vanishes in the joint limit. It is structural for residual networks, transformers, and recurrent networks (RNNs, LSTMs, SSMs), each discretizing the same class of equations, at a named and quantified approximation error. A single deformation parameter $\varepsilon$ unifies all four perspectives (network, tropical algebra, viscous PDE, convex optimization) in a commutative diagram closed under Lipschitz conditions. Quantitative consequences include: the minimax optimal generalization rate $O(n^{-1/(d+2)})$ for fixed $t$; adversarial robustness controlled by $\varepsilon$; backpropagation as the co-state equation of the Hamiltonian system for residual networks (Pontryagin Maximum Principle); scaling exponents consistent with data intrinsic dimension via PDE quadrature; and a closed-form $O(N)$ influence function (softmax attribution weights $\pi_j$) whose entropy landscape undergoes fold bifurcations as $\varepsilon$ increases, each merging attribution basins.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Dynamical Systems (math.DS); Representation Theory (math.RT); Computational Physics (physics.comp-ph)
arXiv:2605.28983 [cs.LG]
(or arXiv:2605.28983v2 [cs.LG] for this version)
Text integral sub licența deschisă a publicației · 244 cuvinte
Subiecte conexe
Deep learning methods for inverse problems using connections between proximal operators and Hamilton-Jacobi equations
arXiv · 14 sept. 2026
Direction-Conditioned Policies via Compositional Subgoal Scoring for Online Goal-Conditioned Reinforcement Learning
arXiv · 16 iun. 2026
News
Quantum Computing Report · 25 iul. 2026
Kolmogorov Regression for Robust Diffusion Policies
arXiv · 17 iun. 2026