aiqbench analysis
This paper proposes a novel Bellman optimality equation specifically for optimizing plasticity in continual reinforcement learning, addressing a gap left by recent formulations of the stability-plasticity tradeoff. By defining plasticity through generalized directed information, the authors establish a theoretical framework that mirrors existing empowerment optimization methods within Markov decision processes. This work advances the fundamental understanding of how agents can dynamically balance adaptability and stability without relying on incidental AI applications.
A Bellman Optimality Equation for Plasticity
arXiv:2609.10776v1 Announce Type: new Abstract: In continual reinforcement learning, carefully managing the stability-plasticity tradeoff remains a core challenge. Recent work by Abel et al. (2025) formalized this dilemma by defining plasticity as the generalized directed information from an agent's observations to its actions, and empowerment as the generalized directed information from its actions to its observations. This formulation successfully reframes the traditional stability-plasticity tradeoff as an empowerment-plasticity tradeoff. However, while extensive literature exists on optimizing for empowerment, there is currently no research addressing the optimization of plasticity under this new definition. This paper presents preliminary work toward optimizing plasticity within Markov decision processes. We show that there exists a Bellman optimality equation for optimizing plasticity similar to previous work for empowerment.
Abstract:In continual reinforcement learning, carefully managing the stability-plasticity tradeoff remains a core challenge. Recent work by Abel et al. (2025) formalized this dilemma by defining plasticity as the generalized directed information from an agent's observations to its actions, and empowerment as the generalized directed information from its actions to its observations. This formulation successfully reframes the traditional stability-plasticity tradeoff as an empowerment-plasticity tradeoff. However, while extensive literature exists on optimizing for empowerment, there is currently no research addressing the optimization of plasticity under this new definition. This paper presents preliminary work toward optimizing plasticity within Markov decision processes. We show that there exists a Bellman optimality equation for optimizing plasticity similar to previous work for empowerment.
Machine Learning (cs.LG)
arXiv:2609.10776 [cs.LG]
(or arXiv:2609.10776v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2609.10776 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 210 words
Related coverage
Offline Deep Q* Estimation with Diffusion Models
arXiv · Aug 17, 2026
Rationalizing Boltzmann Rationality: An Axiomatic Characterization of Entropy-Regularized Policies
arXiv · Jul 21, 2026
Continuous-time reinforcement learning for optimal switching over multiple regimes
arXiv · Jul 31, 2026
Bellman Policy Optimization
arXiv · Sep 15, 2026