analiza aiqbench
DimGrow introduces a memory-efficient method for automated embedding dimension allocation, addressing the scalability limits of traditional SuperNet-based approaches in large feature spaces. By progressively expanding dimensions based on importance scoring rather than exhaustive enumeration, it significantly reduces training memory requirements for recommendation systems. This advancement optimizes resource usage without compromising model performance, making high-dimensional AI architectures more feasible for deployment.
DimGrow: Memory-Efficient Field-level Embedding Dimension Search
arXiv:2505.12683v2 Announce Type: replace Abstract: Key feature fields need bigger embedding dimensionality, others need smaller. This demands automated dimension allocation. Existing approaches, such as pruning or Neural Architecture Search (NAS), require training a memory-intensive SuperNet that enumerates all possible dimension combinations, which is infeasible for large feature spaces. We propose DimGrow, a lightweight approach that eliminates the SuperNet requirement. Starting training model from one dimension per feature field, DimGrow can progressively expand/shrink dimensions via importance scoring. Dimensions grow only when their importance consistently exceed a threshold, ensuring memory efficiency. Experiments on three recommendation datasets verify the effectiveness of DimGrow while it reduces training memory compared to SuperNet-based methods.
Abstract:Key feature fields need bigger embedding dimensionality, others need smaller. This demands automated dimension allocation. Existing approaches, such as pruning or Neural Architecture Search (NAS), require training a memory-intensive SuperNet that enumerates all possible dimension combinations, which is infeasible for large feature spaces. We propose DimGrow, a lightweight approach that eliminates the SuperNet requirement. Starting training model from one dimension per feature field, DimGrow can progressively expand/shrink dimensions via importance scoring. Dimensions grow only when their importance consistently exceed a threshold, ensuring memory efficiency. Experiments on three recommendation datasets verify the effectiveness of DimGrow while it reduces training memory compared to SuperNet-based methods.
This paper is duplicated. As the newest version of this paper is arXiv:2503.09315. In addition, this paper will prevent the newest version of paper during the review process
Machine Learning (cs.LG)
arXiv:2505.12683 [cs.LG]
(or arXiv:2505.12683v2 [cs.LG] for this version)
Text integral sub licența deschisă a publicației · 142 cuvinte
Subiecte conexe
Lattice-Based Cryptography Explained
Quantum Zeitgeist · 15 aug. 2026
Presentation: Postgres for Production Agents: Your Relational Foundation for Enterprise AI
InfoQ · 15 iul. 2026
Humanity’s Last Invention — Richard Socher of Recursive
Latent Space · 14 sept. 2026
Adaptive Parallel Reasoning: The Next Paradigm in Efficient Inference Scaling
BAIR Blog · 8 mai 2026