analiza aiqbench
This reproduction study finds that the Robotic World Model's ensemble uncertainty ranks rollout errors well but badly understates them: one-sigma coverage is 4.6% at 100 steps, against 68% for a calibrated Gaussian. A horizon-specific rescaling fixes one-sigma coverage but not two-sigma, and the authors report two defects in the released evaluation pipeline. The scope is narrow (one robot, one gait, one dataset, no policy trained), so the results are a useful caution about uncertainty-penalized offline RL rather than a general verdict on learned world models.
Right Ranking, Wrong Scale: Reproducing the Robotic World Model's Dynamics Model and Testing the Uncertainty of Its Follow-up
arXiv:2611.02699v1 Announce Type: cross Abstract: We reimplement the dynamics model of the Robotic World Model (RWM) and test the uncertainty estimate that its follow-up, RWM-U, uses to penalize reward in offline policy optimization. Our implementation matches the released model's gradients. On held-out data, training on the model's own rollouts beats one-step training at the preregistered long horizon for all ten episodes, but loses on short one-step windows. RWM-U's ensemble disagreement ranks rollout errors better than forecast step (Spearman rho = 0.90 at 100 steps), yet its spread substantially understates error: one-sigma coverage on the released checkpoint is 4.6% at 100 steps, versus 68% for a calibrated Gaussian. Rolling ensemble members on their own predictions raises coverage to 33%. The implemented squared-error loss on sampled predictions drives each member's variance toward zero. A horizon-specific rescaling restores one-sigma coverage under leave-one-episode-out evaluation, but not two-sigma coverage. We also identify two defects in the released evaluation pipeline. The study covers one robot, one gait and one dataset; it trains no policy.
Abstract:We reimplement the dynamics model of the Robotic World Model (RWM) and test the uncertainty estimate that its follow-up, RWM-U, uses to penalize reward in offline policy optimization. Our implementation matches the released model's gradients. On held-out data, training on the model's own rollouts beats one-step training at the preregistered long horizon for all ten episodes, but loses on short one-step windows. RWM-U's ensemble disagreement ranks rollout errors better than forecast step (Spearman rho = 0.90 at 100 steps), yet its spread substantially understates error: one-sigma coverage on the released checkpoint is 4.6% at 100 steps, versus 68% for a calibrated Gaussian. Rolling ensemble members on their own predictions raises coverage to 33%. The implemented squared-error loss on sampled predictions drives each member's variance toward zero. A horizon-specific rescaling restores one-sigma coverage under leave-one-episode-out evaluation, but not two-sigma coverage. We also identify two defects in the released evaluation pipeline. The study covers one robot, one gait and one dataset; it trains no policy.
Robotics (cs.RO); Machine Learning (cs.LG)
arXiv:2611.02699 [cs.RO]
(or arXiv:2611.02699v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2611.02699 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 259 cuvinte
Subiecte conexe
Uncertainty-Aware Decision Making in Multimodal Large Language Models
arXiv · 19 aug. 2026
Interspecies clock comparison below $5 \times 10^{-18}$ uncertainty with a transportable clock
arXiv · 4 aug. 2026
Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility
arXiv · 5 aug. 2026
An Introduction to Large Language Models for Scientists and Professionals
arXiv · 31 oct. 2026