aiqbench analysis
This study systematically quantifies the significant variability in dynamic cloth manipulation when identical trajectories are applied, revealing discrepancies between real-world physical properties and current simulation capabilities. By releasing a comprehensive dataset of 269 rollouts alongside vision and robot logs, the work provides essential benchmarks for improving the reliability of learned manipulation policies in robotics. The findings highlight that existing simulators fail to reproduce the magnitude and ordering of this variability, pointing to critical gaps in physics-based modeling for soft object interaction.
Same Action, Different Outcome: Variability in Dynamic Cloth Manipulation
arXiv:2610.10801v1 Announce Type: new Abstract: Although cloth is known to exhibit different outcomes under repeated fast dynamic motions, even when the same trajectory is applied, this variability has not yet been systematically characterized. Quantifying it is essential to assess the reliability of learned manipulation policies and the extent to which simulation can reproduce real-world behavior. To study this, we execute the same trajectory ten times across four dynamic tasks, two of which are novel, each tested with three cloths of very different properties and at up to three execution speeds, with a total of 269 recorded rollouts. For all of them, we record small marker positions on the cloth and synchronized stereo camera. We then formalize different metrics to quantify variability, and our results show how it is significant in every test condition, in most cases one to three orders of magnitude above the repeatability inherent to the robot and sensing noise. Our results also show variability is driven mainly by the cloth physical properties but also grows with speed. By replaying all the rollouts in four calibrated modern cloth simulators, we show that none of them can reproduce the variability magnitude we observed in real cloth, nor its ordering. Together with our analysis, we publish the dataset with synchronized OptiTrack, vision and robot logs and their corresponding simulator twins.
Abstract:Although cloth is known to exhibit different outcomes under repeated fast dynamic motions, even when the same trajectory is applied, this variability has not yet been systematically characterized. Quantifying it is essential to assess the reliability of learned manipulation policies and the extent to which simulation can reproduce real-world behavior. To study this, we execute the same trajectory ten times across four dynamic tasks, two of which are novel, each tested with three cloths of very different properties and at up to three execution speeds, with a total of 269 recorded rollouts. For all of them, we record small marker positions on the cloth and synchronized stereo camera. We then formalize different metrics to quantify variability, and our results show how it is significant in every test condition, in most cases one to three orders of magnitude above the repeatability inherent to the robot and sensing noise. Our results also show variability is driven mainly by the cloth physical properties but also grows with speed. By replaying all the rollouts in four calibrated modern cloth simulators, we show that none of them can reproduce the variability magnitude we observed in real cloth, nor its ordering. Together with our analysis, we publish the dataset with synchronized OptiTrack, vision and robot logs and their corresponding simulator twins.
Robotics (cs.RO)
arXiv:2610.10801 [cs.RO]
(or arXiv:2610.10801v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2610.10801 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 308 words
Related coverage

Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · Sep 25, 2026

Reviewing failure modes in science as we prepare for AI researchers
LessWrong · Oct 8, 2026

The Entangled Dimensions of Decision Theory
LessWrong · Jul 30, 2026

RAND's extinction report estimates one side of an inequality
LessWrong · Sep 29, 2026