analiza aiqbench
StraTA addresses the critical challenge of long-horizon decision-making in agentic reinforcement learning by introducing explicit trajectory-level strategies to improve exploration and credit assignment. The framework demonstrates significant gains in sample efficiency and success rates across complex environments like ALFWorld, WebShop, and SciWorld, outperforming current frontier models. This advancement represents a meaningful step forward for LLM-based agents capable of executing multi-step tasks with greater reliability.
StraTA: Incentivizing Agentic Reinforcement Learning with Strategic Trajectory Abstraction
arXiv:2605.06642v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used as interactive agents, but optimizing them for long-horizon decision making remains difficult because current methods are largely purely reactive, which weakens both exploration and credit assignment over extended trajectories. In this work, we present Strategic Trajectory Abstraction (StraTA), a simple framework that introduces an explicit trajectory-level strategy into agentic reinforcement learning (RL). StraTA samples a compact strategy from the initial task state, conditions subsequent actions on that strategy, and trains strategy generation and action execution jointly with a hierarchical GRPO-style rollout design, further enhanced by diverse strategy rollout and critical self-judgment. Experiments on ALFWorld, WebShop, and SciWorld show that StraTA consistently improves both sample efficiency and final performance over strong baselines. StraTA reaches success rates of 93.1% on ALFWorld and 84.2% on WebShop. On SciWorld, StraTA attains a 63.5% overall score, outperforming frontier closed-source models.
Abstract:Large language models (LLMs) are increasingly used as interactive agents, but optimizing them for long-horizon decision making remains difficult because current methods are largely purely reactive, which weakens both exploration and credit assignment over extended trajectories. In this work, we present Strategic Trajectory Abstraction (StraTA), a simple framework that introduces an explicit trajectory-level strategy into agentic reinforcement learning (RL). StraTA samples a compact strategy from the initial task state, conditions subsequent actions on that strategy, and trains strategy generation and action execution jointly with a hierarchical GRPO-style rollout design, further enhanced by diverse strategy rollout and critical self-judgment. Experiments on ALFWorld, WebShop, and SciWorld show that StraTA consistently improves both sample efficiency and final performance over strong baselines. StraTA reaches success rates of 93.1% on ALFWorld and 84.2% on WebShop. On SciWorld, StraTA attains a 63.5% overall score, outperforming frontier closed-source models.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2605.06642 [cs.CL]
(or arXiv:2605.06642v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2605.06642 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 238 cuvinte
Subiecte conexe
GLM-5.3 Sometimes Considers How Train-Time Actions Shape Future Behavior
LessWrong · 15 oct. 2026
The Continual Constitutional Convention: How language models revise the documents that govern them
AI Alignment Forum · 13 oct. 2026
Reflection’s Beam model hits 501 billion parameters
Quantum Zeitgeist · 5 oct. 2026
An unexamined cause of the OpenAI Hugging Face hacking incident: its binary performance metric
LessWrong · 23 sept. 2026