aiqbench analysis
StreamRig addresses the challenge of leveraging multi-camera rig geometry in streaming 3D foundation models, which are typically designed for monocular input. By freezing a pre-trained front-end and training only lightweight modules like the Rig-Resampler and CausalBridge, the framework achieves lower pose drift than non-oracle baselines while maintaining low inference costs. This approach enables efficient, real-time odometry for mobile robots and vehicles using synchronized camera arrays without requiring full model retraining.
StreamRig: Exploiting Intra-Rig Geometry for Streaming Multi-Camera Odometry
arXiv:2609.40244v1 Announce Type: cross Abstract: Mobile robots and vehicles carry synchronized multi-camera rigs, yet many streaming 3D foundation models are designed for monocular input, leaving efficient use of rig geometry a challenge. We present StreamRig, a freeze-and-stream framework that builds causal streaming odometry for calibrated rigs on a frozen multi-view 3D foundation model. The frozen front-end jointly perceives the synchronized views using rig calibration. A Rig-Resampler compresses their features, a CausalBridge applies causal attention with a key-value cache, and a lightweight head regresses rig poses. A periodic re-anchoring protocol supports stable pose estimation over long sequences. Only these modules are trained, 74.6M parameters in total, with relative poses as the sole supervision. Our two-stage training strategy combines group relocalization pretraining with causal rig training to transfer the geometric priors of the frozen front-end and the alignment ability of the pretrained modules to streaming odometry. We evaluate on NCLT, TartanGround, KITTI-360, and our self-collected humanoid-robot dataset ZJH, where training uses only simulation and real-world evaluation is zero-shot. Across all four datasets, StreamRig achieves lower translation and rotation drift than the evaluated non-oracle monocular streaming and rig-aware offline models, while maintaining low inference cost. Ablations and controlled camera-count experiments identify the sources of these gains. We further examine how longer training windows affect inference over longer horizons. Code has been released at https://github.com/WeiYuFei0217/StreamRig.
Abstract:Mobile robots and vehicles carry synchronized multi-camera rigs, yet many streaming 3D foundation models are designed for monocular input, leaving efficient use of rig geometry a challenge. We present StreamRig, a freeze-and-stream framework that builds causal streaming odometry for calibrated rigs on a frozen multi-view 3D foundation model. The frozen front-end jointly perceives the synchronized views using rig calibration. A Rig-Resampler compresses their features, a CausalBridge applies causal attention with a key-value cache, and a lightweight head regresses rig poses. A periodic re-anchoring protocol supports stable pose estimation over long sequences. Only these modules are trained, 74.6M parameters in total, with relative poses as the sole supervision. Our two-stage training strategy combines group relocalization pretraining with causal rig training to transfer the geometric priors of the frozen front-end and the alignment ability of the pretrained modules to streaming odometry. We evaluate on NCLT, TartanGround, KITTI-360, and our self-collected humanoid-robot dataset ZJH, where training uses only simulation and real-world evaluation is zero-shot. Across all four datasets, StreamRig achieves lower translation and rotation drift than the evaluated non-oracle monocular streaming and rig-aware offline models, while maintaining low inference cost. Ablations and controlled camera-count experiments identify the sources of these gains. We further examine how longer training windows affect inference over longer horizons. Code has been released at this https URL.
8 pages, 4 figures, 5 tables. Code: this https URL
Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
arXiv:2609.40244 [cs.CV]
(or arXiv:2609.40244v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2609.40244 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 327 words
Related coverage
G2G: Exploiting Intra-Group Geometry for Inter-Group Pose Estimation
arXiv · Jun 9, 2026
G2G: Exploiting Intra-Group Geometry for Inter-Group Pose Estimation
arXiv · Sep 18, 2026
Build a Multi-Camera 3D Tracking Application with NVIDIA DeepStream 9.1 Skills
NVIDIA · Jul 15, 2026
On-the-Fly Homographies Calibration for Multi-Camera Tracking
arXiv · Sep 17, 2026