aiqbench analysis
This research introduces RELIC, a video planning framework that enhances robot manipulation by refining latent embeddings to capture hidden physical properties like friction and object mass from past failures. By employing rejection-based sampling, the method significantly improves replanning efficiency in both simulation and real-world environments without relying on scene appearance alone. This advancement addresses a critical gap in current generative models that previously discarded valuable information revealed during trial-and-error interactions.
Video Replanning via Latent Embedding Refinement and Rejection-Based Sampling
arXiv:2510.17315v3 Announce Type: replace Abstract: Video planning has emerged as a flexible framework for robot manipulation, in which a generative model predicts a video of task completion, and a downstream module translates the predicted frames into actions. However, existing methods typically ignore information from past interactions, limiting their ability to adapt to latent physical properties that can only be revealed through trial and error, such as whether a door should be pushed or pulled, or how friction affects object dynamics. When a plan fails, these methods usually replan from scratch without leveraging the information revealed by the failure. To address this limitation, we introduce RELIC, REplanning with Latent embedding refInement and Candidate rejection, a video planning framework that adapts to hidden physical properties from test-time failures. RELIC optimizes a latent embedding that captures the environment's hidden physical properties from interaction videos and introduces a rejection-based sampling mechanism that filters out hypotheses inconsistent with prior failures. Across eight tasks in two simulation suites, RELIC consistently reduces the number of replanning steps required for success, and linear probes show that its embedding captures the hidden parameters from the interaction itself rather than from scene appearance. Across four challenging real-world robotic manipulation tasks involving hidden interaction modes, e.g., friction, center of mass, and object mass, RELIC raises the one-shot replanning success rate of a video planning baseline from 30.0% to 63.8% after a single physical interaction.
Abstract:Video planning has emerged as a flexible framework for robot manipulation, in which a generative model predicts a video of task completion, and a downstream module translates the predicted frames into actions. However, existing methods typically ignore information from past interactions, limiting their ability to adapt to latent physical properties that can only be revealed through trial and error, such as whether a door should be pushed or pulled, or how friction affects object dynamics. When a plan fails, these methods usually replan from scratch without leveraging the information revealed by the failure. To address this limitation, we introduce RELIC, REplanning with Latent embedding refInement and Candidate rejection, a video planning framework that adapts to hidden physical properties from test-time failures. RELIC optimizes a latent embedding that captures the environment's hidden physical properties from interaction videos and introduces a rejection-based sampling mechanism that filters out hypotheses inconsistent with prior failures. Across eight tasks in two simulation suites, RELIC consistently reduces the number of replanning steps required for success, and linear probes show that its embedding captures the hidden parameters from the interaction itself rather than from scene appearance. Across four challenging real-world robotic manipulation tasks involving hidden interaction modes, e.g., friction, center of mass, and object mass, RELIC raises the one-shot replanning success rate of a video planning baseline from 30.0% to 63.8% after a single physical interaction.
20 pages. Substantial revision with a new title, updated author list and order, expanded simulation and real-world experiments, and additional analyses. Previously titled "Implicit State Estimation via Video Replanning"; the method is now called RELIC (formerly ISE)
Robotics (cs.RO)
arXiv:2510.17315 [cs.RO]
(or arXiv:2510.17315v3 [cs.RO] for this version)
Full text under the publisher's open licence · 275 words
Related coverage
[AINews] not much happened today
Latent Space · Sep 10, 2026
Compressed Video Aggregator: Content-driven Module for Efficient Micro-Video Recommendation
arXiv · Jul 31, 2026
Introducing Instella-MoE: A State-of-the-Art Fully Open Mixture-of-Experts Language Model
AMD · Jul 24, 2026
World Models and Yann LeCun’s Bet Against Large Language Models
Quantum Zeitgeist · Jul 17, 2026