aiqbench analysis
This research addresses a critical bottleneck in scaling robot perception by introducing FeDepth, a federated learning framework that handles heterogeneous platforms through soft clustering rather than rigid domain separation. By allowing clients to participate across multiple overlapping clusters, the method effectively manages continuous domain transitions common in real-world robotic deployments without transferring raw data. The approach significantly improves robustness for depth estimation tasks compared to standard and clustered baselines, offering a practical path toward efficient distributed training on diverse robot fleets.
FeDepth: Federated Learning for Depth Estimation under Robot Heterogeneity
arXiv:2608.01129v1 Announce Type: new Abstract: Although recent robot perception research emphasizes training on data from diverse environments to improve generalization, most existing methods still rely on centralized learning, which is inefficient and difficult to scale across heterogeneous robot platforms. Federated learning (FL) offers an alternative by enabling distributed training without raw data transfer, but it suffers from severe performance degradation under domain shifts caused by heterogeneity across clients. In real robotic deployments, data distributions often overlap across platforms, environments, and sensing conditions, making it difficult to partition clients into clearly separated domains. However, this characteristic breaks the assumption of clearly separable client domains commonly used in clustered FL. To address this gap in robot perception, particularly in depth estimation, we introduce two realistic and unexplored non-IID scenarios that reflect heterogeneity in terms of platform, environment, and depth distribution. We then propose FeDepth, a descriptor-based clustered FL framework that models client relationships through soft clustering. Unlike hard clustering methods that assume clearly separated clusters, FeDepth allows clients to participate in multiple clusters, capturing continuous and ambiguous domain transitions commonly observed in robotic environments. Extensive experiments demonstrate that FeDepth consistently improves robustness over standard FL and clustered FL baselines across multiple depth estimation architectures, providing a practical and effective solution for federated robot perception. Our project page is available at https://vision3d-lab.github.io/fedepth/.
Abstract:Although recent robot perception research emphasizes training on data from diverse environments to improve generalization, most existing methods still rely on centralized learning, which is inefficient and difficult to scale across heterogeneous robot platforms. Federated learning (FL) offers an alternative by enabling distributed training without raw data transfer, but it suffers from severe performance degradation under domain shifts caused by heterogeneity across clients. In real robotic deployments, data distributions often overlap across platforms, environments, and sensing conditions, making it difficult to partition clients into clearly separated domains. However, this characteristic breaks the assumption of clearly separable client domains commonly used in clustered FL. To address this gap in robot perception, particularly in depth estimation, we introduce two realistic and unexplored non-IID scenarios that reflect heterogeneity in terms of platform, environment, and depth distribution. We then propose FeDepth, a descriptor-based clustered FL framework that models client relationships through soft clustering. Unlike hard clustering methods that assume clearly separated clusters, FeDepth allows clients to participate in multiple clusters, capturing continuous and ambiguous domain transitions commonly observed in robotic environments. Extensive experiments demonstrate that FeDepth consistently improves robustness over standard FL and clustered FL baselines across multiple depth estimation architectures, providing a practical and effective solution for federated robot perception. Our project page is available at this https URL.
Accepted at ECCV 2026. Ganghyeon Lee and Inha Lee contributed equally. Kyungdon Joo is the corresponding author
Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
arXiv:2608.01129 [cs.RO]
(or arXiv:2608.01129v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2608.01129 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 332 words
Related coverage
When Is Shallow Enough? Adaptive Split Federated Learning with Client-Specific Sufficiency Estimation
arXiv · Aug 19, 2026
DB-VIO: Dual-Branch Visual Inertial Odometry with Enhanced Visual-Inertial Representation
arXiv · Jul 27, 2026
PointWalker: Mapless Quadruped Locomotion Directly From Raw LiDAR Point Clouds
arXiv · Oct 14, 2026
RLE-Bench: A Qualifying Exam for Coding Agents as Robot Learning Engineers
arXiv · Sep 29, 2026