aiqbench analysis
This research introduces SVD-Attention, a novel mechanism that factors low-rank embeddings to reduce computational complexity from quadratic to linear while preserving softmax normalization. The resulting SOLAR framework enables efficient lifelong recommendation systems capable of processing large-scale user histories without filtering, achieving state-of-the-art performance in industrial evaluations. By addressing the latency and resource constraints inherent in long-context modeling, this work advances the practical deployment of scalable attention mechanisms for real-world AI applications.
SOLAR: SVD-Optimized Lifelong Attention for Recommendation
arXiv:2603.02561v2 Announce Type: replace-cross Abstract: Attention mechanism remains the defining operator in Transformers since it provides expressive global credit assignment, yet its quadratic cost in sequence length N makes long-context modeling expensive and often forces truncation or other heuristics. Linear attention reduces complexity to O(Nd^2) by reordering computation through kernel feature maps, but this reformulation drops the softmax mechanism and shifts the attention score distribution. Lifelong recommendation requires efficient attention as well, for large-scale sequence modeling with user histories and candidate items under tight latency and resource constraints. We introduce SVD-Attention, a novel attention mechanism, and SOLAR, a set-aware framework built on it for lifelong recommendation. SVD-Attention factorizes low-rank embeddings into r principal components, computes candidate-to-interest scores in compact space, and applies softmax over those scores. Its bilinear reduction is exact on the rank-r reconstruction, while the normalized output approximates token-level softmax with an explicitly bounded residual. The resulting computation reduces the cost from O(N^2d) to O(Ndr), and supports 12,000 behaviors and 3,000 candidates per request without filtering. SOLAR achieves the best among compared methods on RecFlow and MIND, delivers 0.8531 AUC at approximately 19ms 95th-percentile latency in an industrial evaluation, and yields business gains with a 0.68% relative lift in Video Views in the real-world online A/B test. Following this evaluation, SOLAR has been fully deployed in Kuaishou's production recommendation system.
Abstract:Attention mechanism remains the defining operator in Transformers since it provides expressive global credit assignment, yet its quadratic cost in sequence length N makes long-context modeling expensive and often forces truncation or other heuristics. Linear attention reduces complexity to O(Nd^2) by reordering computation through kernel feature maps, but this reformulation drops the softmax mechanism and shifts the attention score distribution. Lifelong recommendation requires efficient attention as well, for large-scale sequence modeling with user histories and candidate items under tight latency and resource constraints. We introduce SVD-Attention, a novel attention mechanism, and SOLAR, a set-aware framework built on it for lifelong recommendation. SVD-Attention factorizes low-rank embeddings into r principal components, computes candidate-to-interest scores in compact space, and applies softmax over those scores. Its bilinear reduction is exact on the rank-r reconstruction, while the normalized output approximates token-level softmax with an explicitly bounded residual. The resulting computation reduces the cost from O(N^2d) to O(Ndr), and supports 12,000 behaviors and 3,000 candidates per request without filtering. SOLAR achieves the best among compared methods on RecFlow and MIND, delivers 0.8531 AUC at approximately 19ms 95th-percentile latency in an industrial evaluation, and yields business gains with a 0.68% relative lift in Video Views in the real-world online A/B test. Following this evaluation, SOLAR has been fully deployed in Kuaishou's production recommendation system.
Information Retrieval (cs.IR); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
arXiv:2603.02561 [cs.IR]
(or arXiv:2603.02561v2 [cs.IR] for this version)
https://doi.org/10.48550/arXiv.2603.02561 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 318 words
Related coverage
SequenceO1: End-to-End Ultra-Long (100K) Sequence Modeling in Recommendation with Low-Rank Caching
arXiv · Sep 9, 2026
Solar Open 2 Technical Report
arXiv · Jul 23, 2026
AI Futures: Racing to Lose, A sermon
LessWrong · Sep 28, 2026
The best Garmin watches of 2026: Expert tested and reviewed
ZDNet · Aug 4, 2026