Agentic vs. Parametric World Models: Hybrid Planning for Reliable Language Agents

arXiv:2606.27806v4 Announce Type: replace Abstract: Language agents, such as ReAct, Reflexion, and RAP, implicitly predict how the world will change as they plan and act, using imagined state transitions to guide their decisions. These predictions are encoded in the agents' backbone LLMs as internal beliefs about the environment, effectively forming an implicit world model for multi-step planning. However, hallucinated state information can become embedded in this world model, leading to persistent inaccuracies that influence subsequent decisions. Traditional parametric world models, such as Dreamer and MuZero, learn environment dynamics through explicit transition models, but their limited model capacity makes them difficult to apply directly to modern language-agent planning. We study this tradeoff in graph-structured planning environments and seek to combine the strengths of both paradigms. We propose Hybrid World-Model Planning (HYBRID-WM), which preserves the language agent's semantic planning capabilities while leveraging a lightweight parametric world model for more accurate transition predictions. Specifically, the parametric model constructs a compact planning skeleton to guide action generation while maintaining consistency between the agent's imagined transitions and the model's predictions. Experiments across four graph-structured planning benchmarks demonstrate that HYBRID-WM consistently improves task success and reduces hallucinated-state propagation with modest computational overhead. These findings highlight the effectiveness of combining semantic planning with lightweight parametric grounding for reliable long-horizon language-agent planning.

Title:Agentic vs. Parametric World Models: Hybrid Planning for Reliable Language Agents

Abstract:Language agents, such as ReAct, Reflexion, and RAP, implicitly predict how the world will change as they plan and act, using imagined state transitions to guide their decisions. These predictions are encoded in the agents' backbone LLMs as internal beliefs about the environment, effectively forming an implicit world model for multi-step planning. However, hallucinated state information can become embedded in this world model, leading to persistent inaccuracies that influence subsequent decisions. Traditional parametric world models, such as Dreamer and MuZero, learn environment dynamics through explicit transition models, but their limited model capacity makes them difficult to apply directly to modern language-agent planning. We study this tradeoff in graph-structured planning environments and seek to combine the strengths of both paradigms. We propose Hybrid World-Model Planning (HYBRID-WM), which preserves the language agent's semantic planning capabilities while leveraging a lightweight parametric world model for more accurate transition predictions. Specifically, the parametric model constructs a compact planning skeleton to guide action generation while maintaining consistency between the agent's imagined transitions and the model's predictions. Experiments across four graph-structured planning benchmarks demonstrate that HYBRID-WM consistently improves task success and reduces hallucinated-state propagation with modest computational overhead. These findings highlight the effectiveness of combining semantic planning with lightweight parametric grounding for reliable long-horizon language-agent planning.

Artificial Intelligence (cs.AI)

arXiv:2606.27806 [cs.AI]

(or arXiv:2606.27806v4 [cs.AI] for this version)

https://doi.org/10.48550/arXiv.2606.27806 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

analiza aiqbench

Agentic vs. Parametric World Models: Hybrid Planning for Reliable Language Agents

Subiecte conexe