analiza aiqbench
This paper presents a controlled study on explicit interaction architectures for dynamical learning, comparing structured causal recurrent units against generic echo-state networks (ESN) under fixed computational budgets. The results indicate that structural inductive bias is task-dependent rather than universally superior, with performance varying significantly between custom nonlinear identification and NARMA10 benchmarks. This work offers a rigorous empirical baseline for evaluating whether specific architectural constraints provide genuine advantages over established reservoir computing methods in stateful interaction modeling.
Explicit Interaction Architectures for Dynamical Learning: A Controlled Study of Structural Inductive Bias
arXiv:2606.19101v2 Announce Type: replace-cross Abstract: We investigate a structure-first approach to dynamical learning in which the organization of stateful interactions is prescribed explicitly rather than left entirely to a generic recurrent parameterization. We introduce causal recurrent units built from an ordered sequence of local, state-modulated transformations. The construction is motivated by wave-based interaction models, but the units studied here do not impose scattering, passivity, or energy-balance constraints. Because fixed recurrent dynamics, designed reservoir topologies, readout-only learning, and recurrent depth are already well established, the empirical question is deliberately narrower: does the proposed interaction organization provide a useful inductive bias under controlled computational conditions? We compare a one-layer structured model, a two-layer structured model, and a generic echo-state network (ESN), all with 12 recurrent states and the same strictly linear ridge readout. Each model family receives the same random-search budget on calibration data that are disjoint from the final test data, after which the selected hyperparameters are frozen. On a custom nonlinear identification task, the one-layer structured model attains a mean validation NMSE of 2.76 x 10^-4, compared with 3.19 x 10^-4 for the two-layer model and 3.94 x 10^-4 for the ESN. On NARMA10 the ordering reverses: the ESN attains 0.312, compared with 0.348 and 0.357 for the one- and two-layer structured models. Thus, the proposed organization can be competitive and advantageous on one task, but it is not universally superior; moreover, recurrent depth does not provide a systematic benefit under matched state dimension. The results support a task-dependent interpretation of structural inductive bias and position the present architecture as a controlled precursor to stronger wave- and system-theoretic constructions.
Abstract:We investigate a structure-first approach to dynamical learning in which the organization of stateful interactions is prescribed explicitly rather than left entirely to a generic recurrent parameterization. We introduce causal recurrent units built from an ordered sequence of local, state-modulated transformations. The construction is motivated by wave-based interaction models, but the units studied here do not impose scattering, passivity, or energy-balance constraints. Because fixed recurrent dynamics, designed reservoir topologies, readout-only learning, and recurrent depth are already well established, the empirical question is deliberately narrower: does the proposed interaction organization provide a useful inductive bias under controlled computational conditions? We compare a one-layer structured model, a two-layer structured model, and a generic echo-state network (ESN), all with 12 recurrent states and the same strictly linear ridge readout. Each model family receives the same random-search budget on calibration data that are disjoint from the final test data, after which the selected hyperparameters are frozen. On a custom nonlinear identification task, the one-layer structured model attains a mean validation NMSE of 2.76 x 10^-4, compared with 3.19 x 10^-4 for the two-layer model and 3.94 x 10^-4 for the ESN. On NARMA10 the ordering reverses: the ESN attains 0.312, compared with 0.348 and 0.357 for the one- and two-layer structured models. Thus, the proposed organization can be competitive and advantageous on one task, but it is not universally superior; moreover, recurrent depth does not provide a systematic benefit under matched state dimension. The results support a task-dependent interpretation of structural inductive bias and position the present architecture as a controlled precursor to stronger wave- and system-theoretic constructions.
14 pages, 5 figures, 2 tables. Substantially revised version with expanded related work, controlled ESN comparisons under matched state dimension and calibration budget, additional NARMA10 experiments, and revised conclusions regarding structural inductive bias and depth
Signal Processing (eess.SP); Machine Learning (cs.LG)
arXiv:2606.19101 [eess.SP]
(or arXiv:2606.19101v2 [eess.SP] for this version)
Text integral sub licența deschisă a publicației · 313 cuvinte
Subiecte conexe
Salesforce after Dreamforce: How the CRM giant can grow beyond its own interface
SiliconANGLE · 19 sept. 2026
News
Quantum Computing Report · 25 iul. 2026
Angles of attack for continual learning safety
LessWrong · 16 iun. 2026
Reinforcement Learning: From Algorithms To Foundation Models
arXiv · 21 iul. 2026