aiqbench analysis
This research introduces IR-LQR, an algorithm that integrates intrinsic rewards and variance regularization into the online linear quadratic regulator framework to enhance uncertainty-driven exploration. By modifying only the cost function, the method achieves optimal worst-case regret rates while remaining computationally efficient compared to iterative search alternatives. The approach is validated through control tasks involving aircraft pitch angle stabilization and unmanned aerial vehicle dynamics.
Optimistic Online LQR via Intrinsic Rewards
arXiv:2603.28938v2 Announce Type: replace-cross Abstract: Optimism in the face of uncertainty is a popular approach to balance exploration and exploitation in reinforcement learning. Here, we consider the online linear quadratic regulator (LQR) problem, i.e., to learn the LQR corresponding to an unknown linear dynamical system by adapting the control policy online based on closed-loop data collected during operation. In this work, we propose Intrinsic Rewards LQR (IR-LQR), an optimistic online LQR algorithm that applies the idea of intrinsic rewards originating from reinforcement learning and the concept of variance regularization to promote uncertainty-driven exploration. IR-LQR typically retains the structure of a standard LQR synthesis problem by only modifying the cost function, resulting in an intuitively pleasing, simple, computationally cheap, and efficient algorithm. This is in contrast to existing optimistic online LQR formulations that rely on more complicated iterative search algorithms or solve computationally demanding optimization problems. We show that IR-LQR achieves the optimal worst-case regret rate of $\sqrt{T}$, and compare it to various state-of-the-art online LQR algorithms via numerical experiments carried out on an aircraft pitch angle control and an unmanned aerial vehicle example.
Abstract:Optimism in the face of uncertainty is a popular approach to balance exploration and exploitation in reinforcement learning. Here, we consider the online linear quadratic regulator (LQR) problem, i.e., to learn the LQR corresponding to an unknown linear dynamical system by adapting the control policy online based on closed-loop data collected during operation. In this work, we propose Intrinsic Rewards LQR (IR-LQR), an optimistic online LQR algorithm that applies the idea of intrinsic rewards originating from reinforcement learning and the concept of variance regularization to promote uncertainty-driven exploration. IR-LQR typically retains the structure of a standard LQR synthesis problem by only modifying the cost function, resulting in an intuitively pleasing, simple, computationally cheap, and efficient algorithm. This is in contrast to existing optimistic online LQR formulations that rely on more complicated iterative search algorithms or solve computationally demanding optimization problems. We show that IR-LQR achieves the optimal worst-case regret rate of $\sqrt{T}$, and compare it to various state-of-the-art online LQR algorithms via numerical experiments carried out on an aircraft pitch angle control and an unmanned aerial vehicle example.
Systems and Control (eess.SY); Machine Learning (cs.LG); Optimization and Control (math.OC)
arXiv:2603.28938 [eess.SY]
(or arXiv:2603.28938v2 [eess.SY] for this version)
https://doi.org/10.48550/arXiv.2603.28938 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 278 words
Related coverage
Curiosity-Critic: Cumulative Prediction Error Improvement as a Tractable Intrinsic Reward for World Model Training
arXiv · Jun 17, 2026
Provable Benefits of Regularization: Fast Rates for Adversarial Imitation Learning
arXiv · Sep 29, 2026
Robust Risk-Sensitive Reinforcement Learning from Corrupted Human Feedback
arXiv · Oct 1, 2026
Adaptive Bayes exactly tracks information over intrinsic time
arXiv · Aug 27, 2026