analiza aiqbench
Simplicial embeddings offer a lightweight geometric inductive bias that enhances sample efficiency and stabilizes training in actor-critic reinforcement learning agents without sacrificing runtime speed. By enforcing sparse, discrete features, this approach effectively addresses the high interaction costs often associated with parallelized environment training. The method demonstrates consistent performance gains across diverse control environments when integrated with standard algorithms like FastTD3, FastSAC, and PPO.
Simplicial Embeddings Improve Sample Efficiency in Actor-Critic Agents
arXiv:2510.13704v2 Announce Type: replace Abstract: Recent works have proposed accelerating the wall-clock training time of actor-critic methods via the use of large-scale environment parallelization; unfortunately, these can sometimes still require large number of environment interactions to achieve a desired level of performance. Noting that well-structured representations can improve the generalization and sample efficiency of deep reinforcement learning (RL) agents, we propose the use of simplicial embeddings: lightweight representation layers that constrain embeddings to simplicial structures. This geometric inductive bias results in sparse and discrete features that stabilize critic bootstrapping and strengthen policy gradients. When applied to FastTD3, FastSAC, and PPO, simplicial embeddings consistently improve sample efficiency and final performance across a variety of continuous- and discrete-control environments, without any loss in runtime speed.
Abstract:Recent works have proposed accelerating the wall-clock training time of actor-critic methods via the use of large-scale environment parallelization; unfortunately, these can sometimes still require large number of environment interactions to achieve a desired level of performance. Noting that well-structured representations can improve the generalization and sample efficiency of deep reinforcement learning (RL) agents, we propose the use of simplicial embeddings: lightweight representation layers that constrain embeddings to simplicial structures. This geometric inductive bias results in sparse and discrete features that stabilize critic bootstrapping and strengthen policy gradients. When applied to FastTD3, FastSAC, and PPO, simplicial embeddings consistently improve sample efficiency and final performance across a variety of continuous- and discrete-control environments, without any loss in runtime speed.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Robotics (cs.RO)
arXiv:2510.13704 [cs.LG]
(or arXiv:2510.13704v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2510.13704 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 215 cuvinte