analiza aiqbench
This arXiv paper is squarely about reinforcement learning: it extends Q-learning to risk-averse finite-horizon MDPs using Markov coherent risk measures and feature-based Q-factor approximation, with a high-probability regret bound. For readers, the practical takeaway is theoretical rather than product-level: the bound scales exponentially in the horizon through the mini-batch size (N^(H-1)), so the approach is provably sound but may be costly for long horizons, and validation is limited to small illustrative problems like an assignment task and a short bandit setting.
Reinforcement Learning with Markov Risk Measures and Multipattern Risk Approximation
arXiv:2605.00654v2 Announce Type: replace-cross Abstract: For a risk-averse finite-horizon Markov Decision Process, we introduce a special class of Markov coherent risk measures, called mini-batch measures. We also define the class of multipattern risk-averse problems that reduce to linear MDPs in the risk-neutral case. We use both concepts in a feature-based $Q$-learning method with multipattern $Q$-factor approximation and we prove a high-probability regret bound of $\widetilde{\mathcal{O}}\big(d^{3/2}HN^{H-1}\sqrt{K}\big)$, where $d$ is the number of features, $H$ is the horizon, $N\ge2$ is the mini-batch size, $K$ is the number of episodes, and $\widetilde{\mathcal{O}}$ hides logarithmic factors. We also propose an economical version of the $Q$-learning method that streamlines the policy evaluation (backward) step. The theoretical results are illustrated on a stochastic assignment problem and a short-horizon multi-armed bandit problem.
Abstract:For a risk-averse finite-horizon Markov Decision Process, we introduce a special class of Markov coherent risk measures, called mini-batch measures. We also define the class of multipattern risk-averse problems that reduce to linear MDPs in the risk-neutral case. We use both concepts in a feature-based $Q$-learning method with multipattern $Q$-factor approximation and we prove a high-probability regret bound of $\widetilde{\mathcal{O}}\big(d^{3/2}HN^{H-1}\sqrt{K}\big)$, where $d$ is the number of features, $H$ is the horizon, $N\ge2$ is the mini-batch size, $K$ is the number of episodes, and $\widetilde{\mathcal{O}}$ hides logarithmic factors. We also propose an economical version of the $Q$-learning method that streamlines the policy evaluation (backward) step. The theoretical results are illustrated on a stochastic assignment problem and a short-horizon multi-armed bandit problem.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Optimization and Control (math.OC); Machine Learning (stat.ML)
arXiv:2605.00654 [cs.LG]
(or arXiv:2605.00654v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2605.00654 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 222 cuvinte
Subiecte conexe
Provably Efficient Federated Reinforcement Learning with Linear Function Approximation and Logarithmic Communication Cost
arXiv · 2 sept. 2026
On the Complexity of Offline Reinforcement Learning with $Q^\star$-Approximation and Partial Coverage
arXiv · 9 iun. 2026
Reinforcement LearningtoHarness Approximation Errors for Long-Time QuantumSimulation
arXiv · 21 aug. 2026
Almost Sure Convergence Rates of Stochastic Approximation and Reinforcement Learning via a Poisson-Moreau Drift
arXiv · 4 nov. 2026