aiqbench analysis
RegionFed addresses a critical bottleneck in federated learning for retail search by operating at the gradient level rather than the parameter level, thereby avoiding the catastrophic collapse seen in transformer models with tied embeddings. This architecture-robust framework adaptively routes regions to optimal personalization strategies while maintaining differential privacy, achieving performance nearly identical to centralized training without data centralization. The approach demonstrates significant gains across diverse model architectures including T5 and RoBERTa, offering a scalable solution for handling data heterogeneity in large-scale AI systems.
RegionFed: Federated Learning for Personalized Query Understanding in Heterogeneous Retail Environments
arXiv:2609.05403v1 Announce Type: cross Abstract: Retail search systems serve diverse geographic regions with distinct query patterns, vocabularies, and product preferences, creating significant data heterogeneity that challenges both privacy-preserving training and model personalization. Federated learning offers a natural solution for privacy, but standard FL methods produce global models that sacrifice regional performance, while existing personalized FL approaches operate at the parameter level and catastrophically collapse on modern transformers (below 10\% accuracy on T5) due to tied embeddings and LayerNorm interactions. We introduce RegionFed, an \textit{architecture-robust} federated learning framework that sidesteps this failure by operating entirely at the gradient level. RegionFed uses the $\ell_2$ conflict between regional and global gradients as a unified signal that (i) diagnoses heterogeneity, (ii) routes each region to the cheapest sufficient personalization strategy, and (iii) adaptively controls personalization strength. Because it treats models as differentiable black boxes, RegionFed deploys on T5-Small, T5-3B, RoBERTa, and CNN with zero code changes, providing large gains on transformers (where parameter-level methods collapse) and consistent improvements on CNNs. Across three public datasets (Amazon ESCI, Amazon Reviews, LEAF-FEMNIST) and four architectures, RegionFed-Meta achieves 92.27\%, closing the gap to the privacy-violating centralized upper bound (Centralized + Regional Weighting: 92.04\%, $\Delta$=0.23pp, within 1$\sigma$) while providing $(\epsilon{\approx}0.60)$-differential privacy and $\mathcal{O}(1/\sqrt{T})$ convergence.
Abstract:Retail search systems serve diverse geographic regions with distinct query patterns, vocabularies, and product preferences, creating significant data heterogeneity that challenges both privacy-preserving training and model personalization. Federated learning offers a natural solution for privacy, but standard FL methods produce global models that sacrifice regional performance, while existing personalized FL approaches operate at the parameter level and catastrophically collapse on modern transformers (below 10\% accuracy on T5) due to tied embeddings and LayerNorm interactions. We introduce RegionFed, an \textit{architecture-robust} federated learning framework that sidesteps this failure by operating entirely at the gradient level. RegionFed uses the $\ell_2$ conflict between regional and global gradients as a unified signal that (i) diagnoses heterogeneity, (ii) routes each region to the cheapest sufficient personalization strategy, and (iii) adaptively controls personalization strength. Because it treats models as differentiable black boxes, RegionFed deploys on T5-Small, T5-3B, RoBERTa, and CNN with zero code changes, providing large gains on transformers (where parameter-level methods collapse) and consistent improvements on CNNs. Across three public datasets (Amazon ESCI, Amazon Reviews, LEAF-FEMNIST) and four architectures, RegionFed-Meta achieves 92.27\%, closing the gap to the privacy-violating centralized upper bound (Centralized + Regional Weighting: 92.04\%, $\Delta$=0.23pp, within 1$\sigma$) while providing $(\epsilon{\approx}0.60)$-differential privacy and $\mathcal{O}(1/\sqrt{T})$ convergence.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2609.05403 [cs.LG]
(or arXiv:2609.05403v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2609.05403 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 297 words
Related coverage
Data Mesh vs. Data Fabric: Key Differences and How the Lakehouse Resolves the Debate
Databricks · Aug 25, 2026
Presentation: From Models to Agents: Building Context-Aware Consumer AI at Scale at DoorDash
InfoQ · Aug 15, 2026
Vertical Advantage: Transforming Industries with Lakebase and Agentic AI
Databricks · Aug 27, 2026
Building multi-Region visualizations with Highcharts in Amazon Quick
AWS · Jul 23, 2026