analiza aiqbench
This research introduces a reinforcement learning framework that enables large language models to continuously self-update their context, allowing agents to maintain performance over long sequences of tasks across varying domains. By validating this approach with Qwen3 models, the study demonstrates significant potential for out-of-distribution generalization, addressing a critical bottleneck for deploying autonomous AI systems in dynamic environments.
Connect the Dots: Training LLMs for Long-Lifecycle Agents with Cross-Domain Generalization Via Reinforcement Learning
arXiv:2606.20002v2 Announce Type: replace-cross Abstract: This work presents a general framework for training large language models (LLMs) to "Connect the Dots" (CoD), a meta-capability required by long-lifecycle agents: as an LLM-based AI agent gets deployed in an environment, it solves a long sequence of tasks while continuously exploring the environment, learning from its own experiences, and iteratively self-updating its context about the environment, thereby achieving progressively better performance on future tasks conditioned on the updated context. Major components of the CoD framework include: (1) algorithm design and infrastructure for end-to-end reinforcement learning (RL) with long rollout sequences interleaving solve-task and update-context episodes; (2) tasks and environments for incentivizing and eliciting the targeted meta-capability in LLMs during training, as well as for faithfully measuring progress during evaluation. We present proof-of-concept implementations of the CoD framework, including a GRPO-style RL algorithm with fine-grained credit assignment, as well as tasks and environments tailored to the targeted meta-capability (rather than domain-specific LLM capabilities or standard task-by-task RL). Empirical results with Qwen3-8B and Qwen3.6-27B validate the efficacy of end-to-end RL training in the CoD setting, and demonstrate the potential for out-of-distribution generalization -- within the training domains, across different domains, and from CoD to Ralph-loop settings -- of the elicited meta-capability. Our investigation of CoD connects several lines of prior work, and opens up new opportunities for advancing LLMs and AI agents. To facilitate further research and applications, we release our implementations at https://github.com/agentscope-ai/Trinity-RFT/tree/research/cod_v2/examples/research_cod.
Abstract:This work presents a general framework for training large language models (LLMs) to "Connect the Dots" (CoD), a meta-capability required by long-lifecycle agents: as an LLM-based AI agent gets deployed in an environment, it solves a long sequence of tasks while continuously exploring the environment, learning from its own experiences, and iteratively self-updating its context about the environment, thereby achieving progressively better performance on future tasks conditioned on the updated context. Major components of the CoD framework include: (1) algorithm design and infrastructure for end-to-end reinforcement learning (RL) with long rollout sequences interleaving solve-task and update-context episodes; (2) tasks and environments for incentivizing and eliciting the targeted meta-capability in LLMs during training, as well as for faithfully measuring progress during evaluation. We present proof-of-concept implementations of the CoD framework, including a GRPO-style RL algorithm with fine-grained credit assignment, as well as tasks and environments tailored to the targeted meta-capability (rather than domain-specific LLM capabilities or standard task-by-task RL). Empirical results with Qwen3-8B and Qwen3.6-27B validate the efficacy of end-to-end RL training in the CoD setting, and demonstrate the potential for out-of-distribution generalization -- within the training domains, across different domains, and from CoD to Ralph-loop settings -- of the elicited meta-capability. Our investigation of CoD connects several lines of prior work, and opens up new opportunities for advancing LLMs and AI agents. To facilitate further research and applications, we release our implementations at this https URL.
arXiv v2 updates: scale up experiments to larger Qwen3.6/3.8 models and new domains; support multi-teacher on-policy distillation; add theoretical study of learning dynamics
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
arXiv:2606.20002 [cs.LG]
(or arXiv:2606.20002v2 [cs.LG] for this version)
Text integral sub licența deschisă a publicației · 277 cuvinte
Subiecte conexe
Vertical Advantage: Transforming Industries with Lakebase and Agentic AI
Databricks · 27 aug. 2026
MedPAIR: Measuring Whether Physicians and AI Agree on What Matters in Medical QA
Snorkel AI Blog · 2 oct. 2026
[AINews] The Future of Latent Space
Latent Space · 25 sept. 2026
Can Cloudflare CEO Matthew Prince save the web from AI?
The Verge · 26 sept. 2026