aiqbench analysis
This research addresses a key scalability bottleneck in multi-modal 3D reconstruction by enabling novel view synthesis from unpaired RGB and thermal images without requiring precise calibration. By leveraging a 3D feed-forward transformer for independent pose estimation and a Procrustes-based alignment strategy, the method achieves competitive thermal synthesis while preserving RGB fidelity. The introduction of a dedicated benchmarking framework provides a necessary standard for evaluating cross-modal consistency in future spatial AI systems.
Unpaired RGB-Thermal Gaussian-Splatting Using Visual Geometric Transformers
arXiv:2606.05491v1 Announce Type: cross Abstract: Multi-modal novel view synthesis (NVS) combining RGB and thermal imagery enables precise 3D scene reconstruction with visual and thermal information. However, existing methods typically rely on precisely calibrated RGB-thermal image pairs or stereo setups, limiting scalability and practical deployment. To address this, we introduce a framework for unpaired RGB-thermal NVS that leverages VGGT, a 3D feed-forward transformer architecture, to independently estimate camera poses for each modality. The pose sets are then aligned using the Procrustes algorithm with a cross-modal feature matcher, enabling joint registration without paired calibration. Building on this alignment, we further propose a multi-modal 3D Gaussian Splatting approach that learns directly from unpaired RGB and thermal images. Experiments on diverse scenes demonstrate that our method achieves competitive performance in thermal view synthesis while maintaining RGB fidelity. Moreover, we show that existing reconstruction approaches can produce modality-specific reconstructions that lack cross-modal consistency. We thus introduce a benchmarking framework to rigorously evaluate both per-modality image synthesis and the multi-modal coherence of reconstructed scenes.
Abstract:Multi-modal novel view synthesis (NVS) combining RGB and thermal imagery enables precise 3D scene reconstruction with visual and thermal information. However, existing methods typically rely on precisely calibrated RGB-thermal image pairs or stereo setups, limiting scalability and practical deployment. To address this, we introduce a framework for unpaired RGB-thermal NVS that leverages VGGT, a 3D feed-forward transformer architecture, to independently estimate camera poses for each modality. The pose sets are then aligned using the Procrustes algorithm with a cross-modal feature matcher, enabling joint registration without paired calibration. Building on this alignment, we further propose a multi-modal 3D Gaussian Splatting approach that learns directly from unpaired RGB and thermal images. Experiments on diverse scenes demonstrate that our method achieves competitive performance in thermal view synthesis while maintaining RGB fidelity. Moreover, we show that existing reconstruction approaches can produce modality-specific reconstructions that lack cross-modal consistency. We thus introduce a benchmarking framework to rigorously evaluate both per-modality image synthesis and the multi-modal coherence of reconstructed scenes.
Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
arXiv:2606.05491 [cs.CV]
(or arXiv:2606.05491v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2606.05491 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 262 words
Related coverage
VG3S: Visual Geometry Grounded Gaussian Splatting for Semantic Occupancy Prediction
arXiv · Jul 23, 2026
Gaussian-LIC2: LiDAR-Inertial-Camera Gaussian Splatting SLAM
arXiv · Aug 6, 2026
SURGE: Sonar-fUsed Reconstruction and localization via image-gated Graph Estimation
arXiv · Oct 7, 2026
GaussianBench: Physics-Fidelity Evaluation for Gaussian Scene Representations
arXiv · Oct 9, 2026