aiqbench analysis
This research addresses the robustness of RGB-D semantic segmentation models by introducing Condition Dropout, a training technique designed to handle sensor failures or occlusions where one modality is missing. By simulating incomplete input scenarios during continued training, the method prevents severe performance degradation while maintaining accuracy when full data is available. The approach offers practical improvements for real-world surveillance and robotics applications relying on multi-modal vision systems.
Toward Reliable RGB-D Semantic Segmentation: Handling Missing Modalities via Condition Dropout
arXiv:2607.20326v1 Announce Type: cross Abstract: RGB-D semantic segmentation has achieved remarkable progress, yet most models assume that RGB and depth are always available. In practice, failures or occlusions of surveillance sensors often remove one modality. Although RGB or depth alone can contain sufficient cues, models trained only on full-modality inputs fail to exploit the remaining modality once one is missing, causing severe degradation. We tackle this issue with a simple continued-training paradigm, \emph{Condition Dropout (ConD)}, which mitigates degradation while preserving full-modality accuracy. Starting from a pretrained RGB-D model, ConD adds a second stage that randomly simulates complete, RGB-missing, and depth-missing inputs, freezes the original encoders, and trains copied encoders with zero-initialized feature injection. Experiments on NYU-Depth V2 and SUN RGB-D show that ConD improves robustness under missing modalities and even yields slight gains when modalities are complete. Our code will be made publicly available upon acceptance.
Abstract:RGB-D semantic segmentation has achieved remarkable progress, yet most models assume that RGB and depth are always available. In practice, failures or occlusions of surveillance sensors often remove one modality. Although RGB or depth alone can contain sufficient cues, models trained only on full-modality inputs fail to exploit the remaining modality once one is missing, causing severe degradation. We tackle this issue with a simple continued-training paradigm, \emph{Condition Dropout (ConD)}, which mitigates degradation while preserving full-modality accuracy. Starting from a pretrained RGB-D model, ConD adds a second stage that randomly simulates complete, RGB-missing, and depth-missing inputs, freezes the original encoders, and trains copied encoders with zero-initialized feature injection. Experiments on NYU-Depth V2 and SUN RGB-D show that ConD improves robustness under missing modalities and even yields slight gains when modalities are complete. Our code will be made publicly available upon acceptance.
Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
arXiv:2607.20326 [cs.CV]
(or arXiv:2607.20326v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2607.20326 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 241 words
Related coverage
[AINews] not much happened today
Latent Space · Sep 10, 2026
Presentation: Multi-Agent Patterns from Spotify’s AI Powered Advertising Platform
InfoQ · Oct 8, 2026
Game Design Creativity evals
LessWrong · Oct 12, 2026
Modernizing and scaling support operations with generative AI on AWS
AWS · Sep 2, 2026