analiza aiqbench
This paper applies Johnson-Lindenstrauss random projections to Byzantine-robust distributed optimization, using projected inner products to weight worker updates by their alignment with a trusted reference gradient. The appeal is that the same projection can cut communication costs and support robustness filtering. Readers should check how the method holds up under adaptive attacks and high-dimensional models, and whether the experiments go beyond small benchmarks.
On Johnson-Lindenstrauss Transforms for Byzantine-Robust Optimization
arXiv:2610.27989v1 Announce Type: new Abstract: The growing scale of data and models in machine learning calls for efficient optimization methods. Distributed optimization addresses this challenge, with compression mechanisms helping to reduce communication costs. A classical tool for dimensionality reduction is the Johnson-Lindenstrauss (JL) transform, which is simple to implement and offers strong geometric guarantees. In particular, JL transforms approximately preserve inner products and pairwise distances between vectors with high probability. In this paper, we exploit this property in Byzantine-robust optimization, where worker messages are weighted according to their alignment with a trusted reference gradient. We further support our theoretical findings with experimental results.
Abstract:The growing scale of data and models in machine learning calls for efficient optimization methods. Distributed optimization addresses this challenge, with compression mechanisms helping to reduce communication costs. A classical tool for dimensionality reduction is the Johnson-Lindenstrauss (JL) transform, which is simple to implement and offers strong geometric guarantees. In particular, JL transforms approximately preserve inner products and pairwise distances between vectors with high probability. In this paper, we exploit this property in Byzantine-robust optimization, where worker messages are weighted according to their alignment with a trusted reference gradient. We further support our theoretical findings with experimental results.
Machine Learning (cs.LG)
arXiv:2610.27989 [cs.LG]
(or arXiv:2610.27989v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.27989 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 192 cuvinte