aiqbench analysis
SimuScene advances 3D reconstruction by integrating physics engines directly into the generative loop to correct geometric errors like interpenetration, rather than treating them as post-hoc fixes. This approach yields simulation-ready scenes that maintain stability under gravity, directly addressing a key bottleneck for robotic manipulation and humanoid control tasks. The method demonstrates state-of-the-art performance on physical stability benchmarks, offering a practical solution for deploying reconstructed environments in real-world robotics applications.
SimuScene: Simulation-Ready Compositional 3D Scene Reconstruction from a Single Image
arXiv:2606.03994v1 Announce Type: cross Abstract: Reconstructing interactive, simulation-ready 3D scenes from a single image is a critical bottleneck for robotic manipulation. While recent single-image lifters recover plausible per-object shapes, composing them yields scenes that collapse under physical simulation due to interpenetrating, hovering, or sinking objects. Existing physics-aware methods address this strictly as a post-hoc layout correction, leaving the underlying geometric errors unresolved. To address this, we introduce SimuScene, a compositional 3D reconstruction pipeline that puts physics in the loop of shape and layout estimation. Rather than using physics merely for layout cleanup, we utilize the physics engine as a diagnostic measurement tool during the generative process itself. By diagnostically simulating reconstructed objects under gravity, we convert penetration and support failures into quantitative correction signals that drive gravity-axis stretching and amodal shape resampling. This physics-informed feedback loop mitigates accumulated reconstruction errors and produces a stable, simulation-ready compositional 3D scene. Extensive experiments demonstrate state-of-the-art performance on physical stability and geometric alignment benchmarks. We further highlight SimuScene's utility by deploying reconstructed environments in humanoid control and robot-arm manipulation tasks.
Abstract:Reconstructing interactive, simulation-ready 3D scenes from a single image is a critical bottleneck for robotic manipulation. While recent single-image lifters recover plausible per-object shapes, composing them yields scenes that collapse under physical simulation due to interpenetrating, hovering, or sinking objects. Existing physics-aware methods address this strictly as a post-hoc layout correction, leaving the underlying geometric errors unresolved. To address this, we introduce SimuScene, a compositional 3D reconstruction pipeline that puts physics in the loop of shape and layout estimation. Rather than using physics merely for layout cleanup, we utilize the physics engine as a diagnostic measurement tool during the generative process itself. By diagnostically simulating reconstructed objects under gravity, we convert penetration and support failures into quantitative correction signals that drive gravity-axis stretching and amodal shape resampling. This physics-informed feedback loop mitigates accumulated reconstruction errors and produces a stable, simulation-ready compositional 3D scene. Extensive experiments demonstrate state-of-the-art performance on physical stability and geometric alignment benchmarks. We further highlight SimuScene's utility by deploying reconstructed environments in humanoid control and robot-arm manipulation tasks.
Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
arXiv:2606.03994 [cs.CV]
(or arXiv:2606.03994v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2606.03994 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 270 words
Related coverage
FIRE3D: Feed-forward Interactive 3D Scene Reconstruction Within A Minute
arXiv · Sep 9, 2026
Is Single-View Mesh Reconstruction Ready for Robotics?
arXiv · Sep 9, 2026
CoDimRecon: Agentic Reconstruction of Sim-Ready 3D Scenes with Deformable Curves, Surfaces, and Volumes
arXiv · Sep 30, 2026
Developing Healthcare Robotics with GPU-Native Medical Physics Simulation
NVIDIA · Jul 28, 2026