analiza aiqbench
This paper introduces GTRL, an offline goal-conditioned reinforcement learning algorithm that improves divide-and-conquer scaling by grounding updates with one-step temporal difference targets to correct stochastic bias and ensure coverage of unreachable state-goal pairs. The method demonstrates superior performance across diverse environments on the OGBench benchmark, offering a robust solution for long-horizon planning in data-limited settings.
GTRL: Grounding Divide-and-Conquer Value Learning with Temporal Differences
arXiv:2609.33259v1 Announce Type: new Abstract: In offline goal-conditioned reinforcement learning (GCRL), divide-and-conquer scales to long horizons by joining two shorter segments at a subgoal. However, under stochastic dynamics, the base case of this rule values the luckiest trajectories through the data. The subgoal must also lie on a shared trajectory, so a state-goal pair that no trajectory connects gets no value update at all. To address both, we present Grounded Transitive RL (GTRL), an offline GCRL value learning algorithm that grounds the divide-and-conquer update with a one-step TD target. Over a single step, TD is correct, as its target averages over the successors and needs no subgoal. GTRL adds this target to the composition rather than replacing it, so every pair receives an update, and the composition still carries the long horizon. GTRL also corrects the bias from hindsight relabeling by reweighting each goal against how reachable it was from other successors. We evaluate our algorithm on nineteen OGBench tasks spanning stochastic, deterministic, and stitching environments, where it achieves the highest average success rate. Code will be released soon.
Abstract:In offline goal-conditioned reinforcement learning (GCRL), divide-and-conquer scales to long horizons by joining two shorter segments at a subgoal. However, under stochastic dynamics, the base case of this rule values the luckiest trajectories through the data. The subgoal must also lie on a shared trajectory, so a state-goal pair that no trajectory connects gets no value update at all. To address both, we present Grounded Transitive RL (GTRL), an offline GCRL value learning algorithm that grounds the divide-and-conquer update with a one-step TD target. Over a single step, TD is correct, as its target averages over the successors and needs no subgoal. GTRL adds this target to the composition rather than replacing it, so every pair receives an update, and the composition still carries the long horizon. GTRL also corrects the bias from hindsight relabeling by reweighting each goal against how reachable it was from other successors. We evaluate our algorithm on nineteen OGBench tasks spanning stochastic, deterministic, and stitching environments, where it achieves the highest average success rate. Code will be released soon.
Machine Learning (cs.LG)
arXiv:2609.33259 [cs.LG]
(or arXiv:2609.33259v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2609.33259 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 268 cuvinte
Subiecte conexe
Zero-Shot Time-Series Question Answering via Decoupled Perception and Reasoning
arXiv · 7 oct. 2026
Lotus: Learning to Understand Time-series by Learning to Forecast
arXiv · 15 oct. 2026
A Divide-and-Conquer Quantum-Selected Configuration Interaction for Evaluating $\pi$-$\pi$ Stacking Interaction Energies in the Benzene Dimer
arXiv · 30 sept. 2026
UnSAMv2: Self-Supervised Learning Enables Segment Anything at Any Granularity
arXiv · 15 oct. 2026