Tsallis Entropy Regularization for Linear Quadratic Regulator and Kullback-Leibler Control

Abstract:Shannon entropy regularization is widely adopted in optimal control due to its ability to promote exploration and enhance robustness, e.g., maximum entropy reinforcement learning known as Soft Actor-Critic. The aim of this paper is to show that formulations based on Tsallis entropy, which is a one-parameter extension of Shannon entropy, retain many of the structural and computational advantages of Shannon-entropy-based approaches while offering additional benefits. In particular, we derive a closed-form solution for the linear quadratic regulator and an efficient computational method for the Kullback-Leibler control problem. We also demonstrate its usefulness in balancing between exploration and sparsity of the obtained control law.

Optimization and Control (math.OC); Machine Learning (cs.LG); Systems and Control (eess.SY)

arXiv:2403.01805 [math.OC]

(or arXiv:2403.01805v2 [math.OC] for this version)

https://doi.org/10.48550/arXiv.2403.01805 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.