analiza aiqbench
Kunlun addresses the critical gap in scaling laws for recommendation systems by introducing architectural optimizations that double efficiency and raise Model FLOPs Utilization from 17% to 37% on NVIDIA B200 GPUs. By combining low-level attention mechanisms with high-level computation skipping, this approach resolves the poor scaling efficiency that previously hindered predictable resource allocation in massive-scale ad models. The architecture's immediate deployment in major Meta Ads systems validates its practical impact on production infrastructure.
Kunlun: Establishing Scaling Laws for Massive-Scale Recommendation Systems through Unified Architecture Design
arXiv:2602.10016v3 Announce Type: replace-cross Abstract: Deriving predictable scaling laws that govern the relationship between model performance and computational investment is crucial for designing and allocating resources in massive-scale recommendation systems. While such laws are established for large language models, they remain challenging for recommendation systems, especially those processing both user history and context features. We identify poor scaling efficiency as the main barrier to predictable power-law scaling, stemming from inefficient modules with low Model FLOPs Utilization (MFU) and suboptimal resource allocation. We introduce Kunlun, a scalable architecture that systematically improves model efficiency and resource allocation. Our low-level optimizations include Generalized Dot-Product Attention (GDPA), Hierarchical Seed Pooling (HSP), and Sliding Window Attention. Our high-level innovations feature Computation Skip (CompSkip) and Event-level Personalization. These advances increase MFU from 17% to 37% on NVIDIA B200 GPUs and double scaling efficiency over state-of-the-art methods. Kunlun is now deployed in major Meta Ads models, delivering significant production impact.
Abstract:Deriving predictable scaling laws that govern the relationship between model performance and computational investment is crucial for designing and allocating resources in massive-scale recommendation systems. While such laws are established for large language models, they remain challenging for recommendation systems, especially those processing both user history and context features. We identify poor scaling efficiency as the main barrier to predictable power-law scaling, stemming from inefficient modules with low Model FLOPs Utilization (MFU) and suboptimal resource allocation. We introduce Kunlun, a scalable architecture that systematically improves model efficiency and resource allocation. Our low-level optimizations include Generalized Dot-Product Attention (GDPA), Hierarchical Seed Pooling (HSP), and Sliding Window Attention. Our high-level innovations feature Computation Skip (CompSkip) and Event-level Personalization. These advances increase MFU from 17% to 37% on NVIDIA B200 GPUs and double scaling efficiency over state-of-the-art methods. Kunlun is now deployed in major Meta Ads models, delivering significant production impact.
Information Retrieval (cs.IR); Artificial Intelligence (cs.AI)
arXiv:2602.10016 [cs.IR]
(or arXiv:2602.10016v3 [cs.IR] for this version)
Text integral sub licența deschisă a publicației · 162 cuvinte
Subiecte conexe
Preprocessing Choices in Fine-Tuning Data Matter: Utility and Demographic Sensitivity in LLM-Based Recommendation
arXiv · 3 nov. 2026
Realizing Scaling Laws in Recommender Systems: A Foundation-Expert Paradigm for Hyperscale Model Deployment
arXiv · 28 iul. 2026
MARS: Multi-Agent Re-ranking for Repeat-Order Food Delivery Recommendation
arXiv · 29 iul. 2026
LlamaRec-LKG-RAG: A Single-Pass, Learnable Knowledge Graph-RAG Framework for LLM-Based Ranking
arXiv · 18 aug. 2026