aiqbench analysis
DimGrow introduces a memory-efficient method for automated embedding dimension allocation, addressing the scalability limits of traditional SuperNet-based approaches in large feature spaces. By progressively expanding dimensions based on importance scoring rather than exhaustive enumeration, it significantly reduces training memory requirements for recommendation systems. This advancement optimizes resource usage without compromising model performance, making high-dimensional AI architectures more feasible for deployment.
DimGrow: Memory-Efficient Field-level Embedding Dimension Search
arXiv:2505.12683v2 Announce Type: replace Abstract: Key feature fields need bigger embedding dimensionality, others need smaller. This demands automated dimension allocation. Existing approaches, such as pruning or Neural Architecture Search (NAS), require training a memory-intensive SuperNet that enumerates all possible dimension combinations, which is infeasible for large feature spaces. We propose DimGrow, a lightweight approach that eliminates the SuperNet requirement. Starting training model from one dimension per feature field, DimGrow can progressively expand/shrink dimensions via importance scoring. Dimensions grow only when their importance consistently exceed a threshold, ensuring memory efficiency. Experiments on three recommendation datasets verify the effectiveness of DimGrow while it reduces training memory compared to SuperNet-based methods.
Abstract:Key feature fields need bigger embedding dimensionality, others need smaller. This demands automated dimension allocation. Existing approaches, such as pruning or Neural Architecture Search (NAS), require training a memory-intensive SuperNet that enumerates all possible dimension combinations, which is infeasible for large feature spaces. We propose DimGrow, a lightweight approach that eliminates the SuperNet requirement. Starting training model from one dimension per feature field, DimGrow can progressively expand/shrink dimensions via importance scoring. Dimensions grow only when their importance consistently exceed a threshold, ensuring memory efficiency. Experiments on three recommendation datasets verify the effectiveness of DimGrow while it reduces training memory compared to SuperNet-based methods.
This paper is duplicated. As the newest version of this paper is arXiv:2503.09315. In addition, this paper will prevent the newest version of paper during the review process
Machine Learning (cs.LG)
arXiv:2505.12683 [cs.LG]
(or arXiv:2505.12683v2 [cs.LG] for this version)
Full text under the publisher's open licence · 142 words
Related coverage
Lattice-Based Cryptography Explained
Quantum Zeitgeist · Aug 15, 2026
Presentation: Postgres for Production Agents: Your Relational Foundation for Enterprise AI
InfoQ · Jul 15, 2026
Humanity’s Last Invention — Richard Socher of Recursive
Latent Space · Sep 14, 2026
Adaptive Parallel Reasoning: The Next Paradigm in Efficient Inference Scaling
BAIR Blog · May 8, 2026