aiqbench analysis
GAPSL addresses a critical bottleneck in parallel split learning by introducing gradient alignment mechanisms to handle severe data heterogeneity on edge devices. By dynamically identifying leader gradients and applying direction-aware regularization, the framework significantly improves convergence speed and accuracy compared to standard benchmarks without requiring client-side model aggregation. This advancement is essential for scaling efficient federated learning across diverse IoT environments where resource constraints are paramount.
GAPSL: A Gradient-Aligned Parallel Split Learning over Data-Heterogeneous Edge Computing Systems
arXiv:2603.18540v2 Announce Type: replace Abstract: The increasing complexity of neural networks poses significant challenges for democratizing federated learning (FL) on resource-constrained edge devices. Parallel split learning (PSL) has emerged as a promising solution by offloading substantial computing workload to a server via model partitioning, shrinking client-side computing load, and eliminating the client-side model aggregation for reduced communication and deployment costs. However, the highly heterogeneous nature of client data in edge computing systems causes aggregation-free PSL to suffer from severe training divergence, stemming from gradient directional inconsistency across clients. To address this challenge, we propose GAPSL, a gradient-aligned PSL framework tailored for data-heterogeneous edge systems, which comprises two key components: leader gradient identification (LGI) and gradient direction alignment (GDA). LGI dynamically selects a set of directionally consistent device gradients to construct a leader gradient as a robust proxy for the global convergence trend. GDA employs a direction-aware regularization to align each client's gradient with the leader gradient, thereby mitigating inter-device gradient directional inconsistency and enhancing model convergence. We evaluate GAPSL on a prototype computing testbed. Extensive experiments demonstrate that GAPSL consistently outperforms state-of-the-art benchmarks in training accuracy, convergence latency, and system robustness under severe data heterogeneity.
Abstract:The increasing complexity of neural networks poses significant challenges for democratizing federated learning (FL) on resource-constrained edge devices. Parallel split learning (PSL) has emerged as a promising solution by offloading substantial computing workload to a server via model partitioning, shrinking client-side computing load, and eliminating the client-side model aggregation for reduced communication and deployment costs. However, the highly heterogeneous nature of client data in edge computing systems causes aggregation-free PSL to suffer from severe training divergence, stemming from gradient directional inconsistency across clients. To address this challenge, we propose GAPSL, a gradient-aligned PSL framework tailored for data-heterogeneous edge systems, which comprises two key components: leader gradient identification (LGI) and gradient direction alignment (GDA). LGI dynamically selects a set of directionally consistent device gradients to construct a leader gradient as a robust proxy for the global convergence trend. GDA employs a direction-aware regularization to align each client's gradient with the leader gradient, thereby mitigating inter-device gradient directional inconsistency and enhancing model convergence. We evaluate GAPSL on a prototype computing testbed. Extensive experiments demonstrate that GAPSL consistently outperforms state-of-the-art benchmarks in training accuracy, convergence latency, and system robustness under severe data heterogeneity.
Machine Learning (cs.LG)
arXiv:2603.18540 [cs.LG]
(or arXiv:2603.18540v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2603.18540 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 282 words
Related coverage
Reflections on unlearning and inoculation
LessWrong · Sep 20, 2026
AutoEncoder-Compressed Parallel Split Learning for Pre-trained Model Fine-Tuning
arXiv · Jul 21, 2026
J++ Lens: Jacobian Filtering Enables More Faithful Workspace Lenses
LessWrong · Oct 8, 2026
GEM Training: How Meta Doubled the Efficiency of Its LLM-Scale Ads Foundation Model
Meta Engineering · Aug 3, 2026