analiza aiqbench
This paper addresses the computational bottleneck of applying Bayesian filtering directly to high-dimensional deep models by introducing AURA, a meta-learning framework that projects parameter evolution into a low-dimensional latent space. By performing efficient Extended Kalman filtering in this compressed representation before reconstructing full parameters, the method enables rapid adaptation to non-stationary environments while preserving model expressiveness. The approach demonstrates significant gains in speed and accuracy for tasks ranging from neural wireless receivers to image classification under distribution shifts.
Online Learning via Learned Latent Bayesian Tracking
arXiv:2609.31559v1 Announce Type: new Abstract: Online learning in non-stationary environments requires models to adapt rapidly from streaming data under strict computational constraints. A principled approach casts online learning as Bayesian state tracking, where model parameters are updated sequentially via Bayesian filtering. However, applying Bayesian filters directly to modern deep models is computationally prohibitive due to the high dimensionality of parameter space, forcing existing methods to rely on restrictive approximations or manually designed low-dimensional subspaces. In this work, we identify the absence of a suitable low-dimensional dynamical representation as the core bottleneck in Bayesian filtering-based online learning. Accordingly, we propose Adaptive Update through Representation Adaptation (AURA), a meta-learning framework that learns offline a low-dimensional latent state-space model governing the evolution of optimal model parameters under distribution shift. Online adaptation is then performed via extended Kalman filtering in this learned latent space followed by reconstruction of the full model parameters through a learned lifting map, enabling efficient single-step online adaptation while preserving model expressiveness. Evaluated on online adaptation of neural wireless receivers under time-varying channels and on non-stationary image classification, AURA shows substantial improvements in adaptation speed, accuracy, and computational efficiency over existing online learning and Bayesian filtering baselines, demonstrating that an adaptation-aware latent geometry is beneficial for effective Bayesian online learning in high-dimensional models.
Abstract:Online learning in non-stationary environments requires models to adapt rapidly from streaming data under strict computational constraints. A principled approach casts online learning as Bayesian state tracking, where model parameters are updated sequentially via Bayesian filtering. However, applying Bayesian filters directly to modern deep models is computationally prohibitive due to the high dimensionality of parameter space, forcing existing methods to rely on restrictive approximations or manually designed low-dimensional subspaces. In this work, we identify the absence of a suitable low-dimensional dynamical representation as the core bottleneck in Bayesian filtering-based online learning. Accordingly, we propose Adaptive Update through Representation Adaptation (AURA), a meta-learning framework that learns offline a low-dimensional latent state-space model governing the evolution of optimal model parameters under distribution shift. Online adaptation is then performed via extended Kalman filtering in this learned latent space followed by reconstruction of the full model parameters through a learned lifting map, enabling efficient single-step online adaptation while preserving model expressiveness. Evaluated on online adaptation of neural wireless receivers under time-varying channels and on non-stationary image classification, AURA shows substantial improvements in adaptation speed, accuracy, and computational efficiency over existing online learning and Bayesian filtering baselines, demonstrating that an adaptation-aware latent geometry is beneficial for effective Bayesian online learning in high-dimensional models.
Machine Learning (cs.LG); Signal Processing (eess.SP)
arXiv:2609.31559 [cs.LG]
(or arXiv:2609.31559v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2609.31559 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 305 cuvinte
Subiecte conexe
Physics-Aware Sparse Learning and Selective Online Adaptation for Euler-Lagrange Robot Dynamics
arXiv · 9 iun. 2026
PIRAMID: Progress and Plans
LessWrong · 27 iul. 2026
Hierarchical Latent Structure Learning through Online Inference
arXiv · 27 iul. 2026
Adaptive Bayesian Online Learning via Expert Aggregation
arXiv · 23 iul. 2026