aiqbench analysis
This paper introduces STRAT, an auxiliary task in deep reinforcement learning where agents learn to predict textual traces of their own state to improve navigation and goal tracking in sparse-reward environments. By combining landmark, route, and survey knowledge without human labeling, the method enhances agent performance and provides interpretable accounts of internal beliefs at every step. The approach effectively compacts state representations and prevents rank collapse across complex grid-world tasks.
State Trace Rationale As Auxiliary Task in Reinforcement Learning
arXiv:2609.36867v1 Announce Type: new Abstract: We propose STRAT, an auxiliary task that trains deep reinforcement learning (RL) agents to predict a short textual trace of their own state. Inspired by human spatial navigation, the description combines landmark, route, and survey knowledge, tracking the agent's position, inventory, goals, and immediate progress. Environment rules generate this text online without human labelling. Our method adds a single auxiliary head to a standard policy. Across 60 sparse-reward XLand-MiniGrid tasks, STRAT solves complex environments where standard RL fails outright, while compacting state representations and preventing rank collapse. Beyond performance gains, the predicted trace provides a readable account of agent beliefs at every step for no extra cost.
Abstract:We propose STRAT, an auxiliary task that trains deep reinforcement learning (RL) agents to predict a short textual trace of their own state. Inspired by human spatial navigation, the description combines landmark, route, and survey knowledge, tracking the agent's position, inventory, goals, and immediate progress. Environment rules generate this text online without human labelling. Our method adds a single auxiliary head to a standard policy. Across 60 sparse-reward XLand-MiniGrid tasks, STRAT solves complex environments where standard RL fails outright, while compacting state representations and preventing rank collapse. Beyond performance gains, the predicted trace provides a readable account of agent beliefs at every step for no extra cost.
Artificial Intelligence (cs.AI)
arXiv:2609.36867 [cs.AI]
(or arXiv:2609.36867v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2609.36867 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 201 words
Related coverage
[AINews] not much happened today
Latent Space · Oct 3, 2026
[AINews] Opus 5.5 is good at explainer videos
Latent Space · Sep 29, 2026
Rationale-Guided Policy Optimization: Learning to Reason with Adaptive Rationale Scaffolding
arXiv · Oct 7, 2026
[AINews] The Future of Latent Space
Latent Space · Sep 25, 2026