aiqbench analysis
This research introduces Picasso, a novel pipeline that integrates physics constraints and holistic reasoning to generate geometrically accurate yet physically plausible 3D scene reconstructions from noisy sensor data. By leveraging rejection sampling guided by inferred contact graphs, the method effectively prevents object interpenetration and unstable equilibrium, addressing critical gaps in digital twin simulations for robotics. The accompanying dataset and benchmark metrics provide a new standard for evaluating physical plausibility alongside geometric accuracy in multi-object environments.
Picasso: Holistic Scene Reconstruction with Physics-Constrained Sampling
arXiv:2602.08058v4 Announce Type: replace-cross Abstract: In the presence of occlusions and measurement noise, geometrically accurate scene reconstructions -- which fit the sensor data -- can still be physically incorrect. For instance, when estimating the poses and shapes of objects in the scene and importing the resulting estimates into a simulator, small errors might translate to implausible configurations including object interpenetration or unstable equilibrium. This makes it difficult to predict the dynamic behavior of the scene using a digital twin, an important step in simulation-based planning and control of contact-rich behaviors. In this paper, we posit that object pose and shape estimation requires reasoning holistically over the scene (instead of reasoning about each object in isolation), accounting for object interactions and physical plausibility. Towards this goal, our first contribution is Picasso, a physics-constrained reconstruction pipeline that builds multi-object scene reconstructions by considering geometry, non-penetration, and physics. Picasso relies on a fast rejection sampling method that reasons over multi-object interactions, leveraging an inferred object contact graph to guide samples. Second, we propose the Picasso dataset, a collection of 10 contact-rich real-world scenes with ground truth annotations, as well as a metric to quantify physical plausibility, which we open-source as part of our benchmark. Finally, we provide an extensive evaluation of Picasso on our newly introduced dataset and on the YCB-V dataset, and show it largely outperforms the state of the art while providing reconstructions that are both physically plausible and more aligned with human intuition.
Abstract:In the presence of occlusions and measurement noise, geometrically accurate scene reconstructions -- which fit the sensor data -- can still be physically incorrect. For instance, when estimating the poses and shapes of objects in the scene and importing the resulting estimates into a simulator, small errors might translate to implausible configurations including object interpenetration or unstable equilibrium. This makes it difficult to predict the dynamic behavior of the scene using a digital twin, an important step in simulation-based planning and control of contact-rich behaviors. In this paper, we posit that object pose and shape estimation requires reasoning holistically over the scene (instead of reasoning about each object in isolation), accounting for object interactions and physical plausibility. Towards this goal, our first contribution is Picasso, a physics-constrained reconstruction pipeline that builds multi-object scene reconstructions by considering geometry, non-penetration, and physics. Picasso relies on a fast rejection sampling method that reasons over multi-object interactions, leveraging an inferred object contact graph to guide samples. Second, we propose the Picasso dataset, a collection of 10 contact-rich real-world scenes with ground truth annotations, as well as a metric to quantify physical plausibility, which we open-source as part of our benchmark. Finally, we provide an extensive evaluation of Picasso on our newly introduced dataset and on the YCB-V dataset, and show it largely outperforms the state of the art while providing reconstructions that are both physically plausible and more aligned with human intuition.
Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Robotics (cs.RO); Systems and Control (eess.SY)
arXiv:2602.08058 [cs.CV]
(or arXiv:2602.08058v4 [cs.CV] for this version)
Full text under the publisher's open licence · 261 words
Related coverage
The Machines that Make the Machines
NVIDIA · Oct 7, 2026
[AINews] not much happened today
Latent Space · Sep 18, 2026
Demonstration Synthesis from a Single Scan via Gaussian Splatting for Visuomotor Policy Learning
arXiv · Sep 21, 2026
AI-Driven Neural Surrogates for In Silico Design of Cognitive-Affective Neuromodulation Targets
arXiv · Sep 24, 2026