analiza aiqbench
This research introduces KE-DRL, an offline multi-dimensional distributional reinforcement learning framework that utilizes Hilbert space mappings to efficiently estimate value distributions without relying on computationally expensive Wasserstein metrics. By establishing theoretical guarantees for the Bellman operator under Matern kernels, the method offers a robust solution for risk evaluation in complex decision-making scenarios with continuous state-action variables. The work advances fundamental machine learning theory by demonstrating how embedding-based approaches can simplify off-policy evaluation while maintaining accuracy under mild assumptions like Lipschitz continuity.
Vector-Valued Distributional Reinforcement Learning Policy Evaluation: A Hilbert Space Embedding Approach
arXiv:2601.18952v2 Announce Type: replace Abstract: We propose an (offline) multi-dimensional distributional reinforcement learning framework (KE-DRL) that leverages Hilbert space mappings to estimate the kernel mean embedding of the multi-dimensional value distribution under a proposed target policy. In our setting, the state-action variables are multi-dimensional and continuous. By mapping probability measures into a reproducing kernel Hilbert space via kernel mean embeddings, our method replaces Wasserstein metrics with an integral probability metric. This enables efficient estimation in multi-dimensional state-action spaces and reward settings, where direct computation of Wasserstein distances is computationally challenging. Theoretically, we establish contraction properties of the distributional Bellman operator under our proposed metric involving the Matern family of kernels and provide uniform convergence guarantees. Simulations and empirical results demonstrate robust off-policy evaluation and recovery of the kernel mean embedding under mild assumptions, namely, Lipschitz continuity and boundedness of the kernels, highlighting the potential of embedding-based approaches in complex real-world decision-making scenarios and risk evaluation.
Abstract:We propose an (offline) multi-dimensional distributional reinforcement learning framework (KE-DRL) that leverages Hilbert space mappings to estimate the kernel mean embedding of the multi-dimensional value distribution under a proposed target policy. In our setting, the state-action variables are multi-dimensional and continuous. By mapping probability measures into a reproducing kernel Hilbert space via kernel mean embeddings, our method replaces Wasserstein metrics with an integral probability metric. This enables efficient estimation in multi-dimensional state-action spaces and reward settings, where direct computation of Wasserstein distances is computationally challenging. Theoretically, we establish contraction properties of the distributional Bellman operator under our proposed metric involving the Matern family of kernels and provide uniform convergence guarantees. Simulations and empirical results demonstrate robust off-policy evaluation and recovery of the kernel mean embedding under mild assumptions, namely, Lipschitz continuity and boundedness of the kernels, highlighting the potential of embedding-based approaches in complex real-world decision-making scenarios and risk evaluation.
Machine Learning (cs.LG); Methodology (stat.ME); Machine Learning (stat.ML)
arXiv:2601.18952 [cs.LG]
(or arXiv:2601.18952v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2601.18952 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 247 cuvinte
Subiecte conexe
Continual learning might make your blocking monitors nearly useless
AI Alignment Forum · 24 sept. 2026
Distributional Soft Bellman Operator under the Cram\'er Geometry
arXiv · 21 iul. 2026
QnRL: Quantum-Native Reinforcement Learning
arXiv · 9 iun. 2026
When Partial Credit Opposes Completion
Snorkel AI Blog · 15 oct. 2026