aiqbench analysis
LightSplit addresses the dual challenges of communication overhead and privacy leakage in split learning by applying fixed orthogonal random projections at the network cut layer. This technique acts as an information bottleneck that irreversibly discards specific instance details, effectively resisting reconstruction attacks while maintaining high model accuracy across diverse benchmarks like image classification and language fine-tuning. The approach remains lightweight for edge deployment without requiring changes to existing server-side architectures or adding trainable client components.
LightSplit: Practical Privacy-Preserving Split Learning via Orthogonal Projections (Full Version)
From arXiv
Full textarXiv:2605.13265v2 Announce Type: replace Abstract: Split learning (SL) enables collaborative training by partitioning a neural network across clients and a central server. However, the cut-layer interface introduces a key challenge: high-dimensional activations incur substantial communication overhead while exposing representations vulnerable to reconstruction attacks. Existing approaches typically address efficiency or privacy in isolation. We propose LightSplit, which limits information exposure and reduces communication overhead via a lightweight fixed orthogonal random projection at the cut layer.
Based on Shannon's information theory, this projection acts as an information bottleneck that restricts instance-specific information and suppresses exploitable per-sample signals. With low-dimensional projections sent instead of raw activations, the server operates on lifted representations without requiring changes to existing SL architectures. Adding no trainable client components, LightSplit remains lightweight and suitable for edge devices while preserving end-to-end differentiability via exact gradient propagation. As the projection is non-invertible, part of the original representation is irreversibly discarded at the client, so LightSplit reduces the information available for reconstruction.
We evaluate LightSplit on seven benchmarks spanning image classification, tabular data, and language-model fine-tuning, under IID and non-IID client data. At up to 32x compression it retains at least 94% of baseline accuracy on CIFAR-10, FashionMNIST, GTSRB, and UCI HAR, 86% on CIFAR-100, and 91% on average under non-IID client data; at 8x it retains 92% of ImageNet-1k top-1 and raises GPT-2 perplexity by at most 1.15x. Against state-of-the-art reconstruction attacks LightSplit resists inversion; the strongest, co-trained attack loses up to 87.3% of its reconstruction SSIM, and subject re-identification on UCI HAR drops by more than half.
Abstract:Split learning (SL) enables collaborative training by partitioning a neural network across clients and a central server. However, the cut-layer interface introduces a key challenge: high-dimensional activations incur substantial communication overhead while exposing representations vulnerable to reconstruction attacks. Existing approaches typically address efficiency or privacy in isolation. We propose LightSplit, which limits information exposure and reduces communication overhead via a lightweight fixed orthogonal random projection at the cut layer. Based on Shannon's information theory, this projection acts as an information bottleneck that restricts instance-specific information and suppresses exploitable per-sample signals.
With low-dimensional projections sent instead of raw activations, the server operates on lifted representations without requiring changes to existing SL architectures. Adding no trainable client components, LightSplit remains lightweight and suitable for edge devices while preserving end-to-end differentiability via exact gradient propagation. As the projection is non-invertible, part of the original representation is irreversibly discarded at the client, so LightSplit reduces the information available for reconstruction. We evaluate LightSplit on seven benchmarks spanning image classification, tabular data, and language-model fine-tuning, under IID and non-IID client data.
At up to 32x compression it retains at least 94% of baseline accuracy on CIFAR-10, FashionMNIST, GTSRB, and UCI HAR, 86% on CIFAR-100, and 91% on average under non-IID client data; at 8x it retains 92% of ImageNet-1k top-1 and raises GPT-2 perplexity by at most 1.15x. Against state-of-the-art reconstruction attacks LightSplit resists inversion; the strongest, co-trained attack loses up to 87.3% of its reconstruction SSIM, and subject re-identification on UCI HAR drops by more than half.
Machine Learning (cs.LG); Cryptography and Security (cs.CR)
arXiv:2605.13265 [cs.LG]
(or arXiv:2605.13265v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2605.13265 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 349 words