aiqbench analysis
This research addresses the critical challenge of resource-constrained adaptation for robots operating in non-stationary environments by optimizing when to perform computationally expensive policy updates. By formulating an online, regret-guided scheduling rule within time-varying Markov decision processes, the study provides a theoretical bound that allows agents to adaptively allocate limited energy and compute budgets efficiently. The approach demonstrates superior performance over static baselines in dynamic scenarios ranging from simulated Mars-rover navigation to real-world quadrotor obstacle avoidance.
When Should a Robot Replan? Regret-Guided Update Scheduling in Time-Varying MDPs
arXiv:2606.16972v1 Announce Type: new Abstract: Robots operating in non-stationary environments must continually adapt their policies as the dynamics drift, but onboard energy and compute budgets cap how often a full state estimation and re-planning step can be performed. This raises a question: \emph{when}, along a horizon, should a robot spend its limited budget? We formulate this problem in time-varying Markov decision processes (TVMDPs) with a known bound on the rate of transition drift. We model execution as a \emph{skip-update} scheme in which, at chosen update times, the agent estimates the transition kernel by maximum likelihood and computes a finite-horizon policy, and between updates reuses this policy under a propagated state estimate. We analyze the dynamic regret of this scheme and show how it grows during skip intervals in terms of the properties of the TVMDP and the skip lengths; the resulting bound answers the opening question via an online, regret-guided update rule that allocates the budget adaptively. We evaluate the rule in a simulated Mars-rover navigation task with time-varying slip dynamics and on a Crazyflie quadrotor in indoor obstacle fields. Adaptive allocation outperforms other budgeted baselines.
Title:When Should a Robot Replan? Regret-Guided Update Scheduling in Time-Varying MDPs
Abstract:Robots operating in non-stationary environments must continually adapt their policies as the dynamics drift, but onboard energy and compute budgets cap how often a full state estimation and re-planning step can be performed. This raises a question: \emph{when}, along a horizon, should a robot spend its limited budget? We formulate this problem in time-varying Markov decision processes (TVMDPs) with a known bound on the rate of transition drift. We model execution as a \emph{skip-update} scheme in which, at chosen update times, the agent estimates the transition kernel by maximum likelihood and computes a finite-horizon policy, and between updates reuses this policy under a propagated state estimate. We analyze the dynamic regret of this scheme and show how it grows during skip intervals in terms of the properties of the TVMDP and the skip lengths; the resulting bound answers the opening question via an online, regret-guided update rule that allocates the budget adaptively. We evaluate the rule in a simulated Mars-rover navigation task with time-varying slip dynamics and on a Crazyflie quadrotor in indoor obstacle fields. Adaptive allocation outperforms other budgeted baselines.
Robotics (cs.RO); Systems and Control (eess.SY)
arXiv:2606.16972 [cs.RO]
(or arXiv:2606.16972v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2606.16972 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 289 words
Related coverage
Dyna-2: General Purpose Physical Agents with Hour-Long Autonomy — Jason Ma, Dyna Robotics
Latent Space · Oct 13, 2026
Post-Train NVIDIA Cosmos 3 Edge for On-Device Robot Control
NVIDIA · Aug 19, 2026
[AINews] not much happened today
Latent Space · Sep 10, 2026
RoboCoach: World Models as Active Coaches for Compositional Robot Skills
arXiv · Oct 1, 2026