aiqbench analysis
This research addresses a critical gap in robotic perception by enabling open-vocabulary 3D semantic segmentation using only depth data, thereby preventing the disclosure of sensitive visual information. The proposed UTTO framework leverages structured predictive uncertainty to refine model outputs without retraining frozen backbones, offering a practical solution for privacy-constrained indoor environments. By validating improvements on standard datasets and demonstrating real-robot applicability, the work establishes a robust pathway for deploying AI-driven robots in scenarios where RGB imagery is prohibited.
Depth-Only Open-Vocabulary 3D Semantic Segmentation For Privacy-Preserving Robotic Applications
arXiv:2607.00978v2 Announce Type: replace-cross Abstract: Privacy-preserving perception is increasingly important for robotic systems operating in real-world indoor environments, yet it remains underexplored in open-vocabulary 3D semantic segmentation. We study this problem under an RGB-prohibited deployment setting motivated by scene-specific visual information disclosure, where real RGB observations are unavailable during scene acquisition and fusion. To reflect this deployment constraint on existing 3D datasets, we adopt a stricter depth-only evaluation protocol that re-runs scene fusion without RGB and exposes only the resulting depth-derived geometry to the segmentation pipeline. This constraint removes appearance cues that are often critical for open-vocabulary recognition, making depth-only predictions more uncertain and less reliable. To address this challenge, we propose UTTO, a model-agnostic uncertainty-guided test-time optimization framework that uses structured predictive uncertainty as a reliability signal to refine predictions from frozen open-vocabulary 3D backbones. Experiments across ScanNet and Matterport3D demonstrate consistent improvements over multiple depth-only backbones. Privacy recoverability analyses and a real-robot semantic goal grounding case study further support the proposed privacy-constrained setting and applicability.
Abstract:Privacy-preserving perception is increasingly important for robotic systems operating in real-world indoor environments, yet it remains underexplored in open-vocabulary 3D semantic segmentation. We study this problem under an RGB-prohibited deployment setting motivated by scene-specific visual information disclosure, where real RGB observations are unavailable during scene acquisition and fusion. To reflect this deployment constraint on existing 3D datasets, we adopt a stricter depth-only evaluation protocol that re-runs scene fusion without RGB and exposes only the resulting depth-derived geometry to the segmentation pipeline. This constraint removes appearance cues that are often critical for open-vocabulary recognition, making depth-only predictions more uncertain and less reliable. To address this challenge, we propose UTTO, a model-agnostic uncertainty-guided test-time optimization framework that uses structured predictive uncertainty as a reliability signal to refine predictions from frozen open-vocabulary 3D backbones. Experiments across ScanNet and Matterport3D demonstrate consistent improvements over multiple depth-only backbones. Privacy recoverability analyses and a real-robot semantic goal grounding case study further support the proposed privacy-constrained setting and applicability.
Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
arXiv:2607.00978 [cs.CV]
(or arXiv:2607.00978v2 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2607.00978 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 259 words
Related coverage
Instance-Enriched Semantic Maps for Visual Language Navigation
arXiv · Jul 20, 2026
Privacy-Preserving Semantic Segmentation from High-Resolution Depth and Ultra-Low-Resolution RGB
arXiv · Sep 24, 2026
Lang3DSeg: Annotation-Free Open-Vocabulary 3D Segmentation with Point Transformers
arXiv · Oct 2, 2026
Standalone DINOv3 for Training-Free Open-Vocabulary Semantic Segmentation in Remote Sensing
arXiv · Aug 5, 2026