aiqbench analysis
EgoPhys advances computer vision by learning generalizable physics models for deformable objects directly from egocentric video without requiring specialized sensors. By distilling inverse-physics solutions into a compact codebook, the framework enables zero-shot prediction of dense spring stiffness fields and facilitates real-to-sim transfer for robotic manipulation tasks involving fabrics and elastic materials. This work addresses critical gaps in simulating complex dynamics using only RGB data, offering scalable pathways for embodied AI systems to understand physical interactions from first-person perspectives.
EgoPhys: Learning Generalizable Physics Models of Deformable Objects from Egocentric Video
arXiv:2606.16202v1 Announce Type: cross Abstract: Humans naturally understand object physics through everyday interactions, but faithfully predicting complex deformable dynamics, such as elastic materials and fabrics, remains a major challenge for computer vision and robotics. We present EgoPhys, a framework that constructs deformable physical digital twins from egocentric RGB-only video using generalizable priors. EgoPhys overcomes the limitations of existing methods to enable controllable deformable digital twin generation from egocentric videos by distilling per-object inverse-physics solutions into a compact codebook, enabling prediction of dense spring stiffness fields for unseen objects without per-spring test-time optimization. Trained with generalizable priors from diverse egocentric interactions, EgoPhys outperforms baselines in reconstruction, future prediction, and zero-shot generalization. To support training and evaluation, we curate an egocentric interaction dataset covering diverse deformable objects, scenes, and manipulation styles. We deploy EgoPhys on a real xArm6 robot, demonstrating that a digital twin initialized from a single egocentric human play video can serve as an internal world representation to aid in deformable-object planning, highlighting egocentric RGB observations as a scalable path toward real-to-sim pipelines.
Abstract:Humans naturally understand object physics through everyday interactions, but faithfully predicting complex deformable dynamics, such as elastic materials and fabrics, remains a major challenge for computer vision and robotics. We present EgoPhys, a framework that constructs deformable physical digital twins from egocentric RGB-only video using generalizable priors. EgoPhys overcomes the limitations of existing methods to enable controllable deformable digital twin generation from egocentric videos by distilling per-object inverse-physics solutions into a compact codebook, enabling prediction of dense spring stiffness fields for unseen objects without per-spring test-time optimization. Trained with generalizable priors from diverse egocentric interactions, EgoPhys outperforms baselines in reconstruction, future prediction, and zero-shot generalization. To support training and evaluation, we curate an egocentric interaction dataset covering diverse deformable objects, scenes, and manipulation styles. We deploy EgoPhys on a real xArm6 robot, demonstrating that a digital twin initialized from a single egocentric human play video can serve as an internal world representation to aid in deformable-object planning, highlighting egocentric RGB observations as a scalable path toward real-to-sim pipelines.
Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Robotics (cs.RO)
arXiv:2606.16202 [cs.CV]
(or arXiv:2606.16202v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2606.16202 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 270 words
Related coverage
NVIDIA Research Advances Robotics From Simulation to the Real World
NVIDIA · May 28, 2026
Learning Implicit Constitutive Laws for Dynamic 3D Gaussian Splatting from Monocular Videos
arXiv · Aug 25, 2026
DLO-Lab: Benchmarking Deformable Linear Object Manipulations with Differentiable Physics
arXiv · Jun 4, 2026
PhySGA: A Physics-Space Grounded Agentic Framework for Generalizable and Interactive 3D Composition
arXiv · Oct 15, 2026