aiqbench analysis
This research advances robot learning by introducing an end-to-end pipeline that reconstructs articulated 3D assets from static video, bridging the sim-to-real gap for complex kinematic structures. By combining photorealistic Gaussian splats with mesh-based geometry and automated joint suggestions, it enables efficient deployment of high-fidelity environments in simulation platforms like NVIDIA Isaac Sim.
RORA: Realistic Object Reconstruction with Articulation
arXiv:2608.04842v1 Announce Type: new Abstract: Replicating real-world environments into simulation by realistic visual representation like NeRF and 3D Gaussian Splatting (3DGS) has emerged as an effective strategy to reduce the sim-to-real gap in robot learning. However, implementing object articulation during the real-to-sim process is still a challenging task. Existing motion tracking or learning based articulation methods shows low success rates on complex kinematic structures having multiple joints. Furthermore, those methods require scan of dynamic motion of objects, which makes reconstruction process much complicated. In this work, we propose the first end-to-end pipeline that reconstructs simulation-ready assets with accurate articulation from a single static object video input through suggestion based human-in-the-loop process. Our approach exports a hybrid representation combining 3DGS for photorealistic rendering and mesh-based geometry for physical interaction. In the reconstruction process, our pipeline performs convex decomposition followed by user grouping for intuitive part segmentation, subsequently binding 3D Gaussians to the corresponding mesh parts. An Automatic Joint Suggestion Algorithm then calculates candidate joint axes from local boundary geometries and presents them to users for efficient articulated asset reconstruction. We have shown that our method achieves precise articulation results on partnet-mobility-v0 dataset and real objects. Additionally we presented a potential usage of our framework on robot learning, deploying the reconstructed assets in Unreal Engine and NVIDIA Isaac Sim, demonstrating real-time dexterous hand manipulation tasks.
Abstract:Replicating real-world environments into simulation by realistic visual representation like NeRF and 3D Gaussian Splatting (3DGS) has emerged as an effective strategy to reduce the sim-to-real gap in robot learning. However, implementing object articulation during the real-to-sim process is still a challenging task. Existing motion tracking or learning based articulation methods shows low success rates on complex kinematic structures having multiple joints. Furthermore, those methods require scan of dynamic motion of objects, which makes reconstruction process much complicated. In this work, we propose the first end-to-end pipeline that reconstructs simulation-ready assets with accurate articulation from a single static object video input through suggestion based human-in-the-loop process. Our approach exports a hybrid representation combining 3DGS for photorealistic rendering and mesh-based geometry for physical interaction. In the reconstruction process, our pipeline performs convex decomposition followed by user grouping for intuitive part segmentation, subsequently binding 3D Gaussians to the corresponding mesh parts. An Automatic Joint Suggestion Algorithm then calculates candidate joint axes from local boundary geometries and presents them to users for efficient articulated asset reconstruction. We have shown that our method achieves precise articulation results on partnet-mobility-v0 dataset and real objects. Additionally we presented a potential usage of our framework on robot learning, deploying the reconstructed assets in Unreal Engine and NVIDIA Isaac Sim, demonstrating real-time dexterous hand manipulation tasks.
This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible
Robotics (cs.RO); Graphics (cs.GR)
arXiv:2608.04842 [cs.RO]
(or arXiv:2608.04842v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2608.04842 arXiv-issued DOI via DataCite (pending registration)
Full text under the publisher's open licence · 263 words
Related coverage
StructureGS: Structure-aware Gaussian Splatting for Articulated Object Reconstruction
arXiv · Jul 30, 2026
Articulated Object Reconstruction from Rest-State Observation
arXiv · Jul 31, 2026
Large to Small Model Stitching Destroys What You Built It to Carry Along the Fit
LessWrong · Oct 11, 2026
Instruct-Particulate: Scaling Feed-Forward 3D Object Articulation with Kinematic Control
arXiv · Jun 16, 2026