analiza aiqbench
This arXiv paper treats regularization in knowledge graph embedding as an under-studied lever, finding that overfitting is common across model families and proposing Sparse Peak Regularization as a model-agnostic alternative to N3, DURA, and ER. The results are benchmark-based on eight datasets, so readers should check whether the gains hold under their own tasks and tuning budgets. For ML practitioners working on link prediction, the main takeaway is that regularizer choice deserves as much attention as architecture.
Rethinking Generalization in Knowledge Graph Embedding: An Empirical Study and Sparse Peak Regularization
arXiv:2505.23442v2 Announce Type: replace Abstract: Knowledge graphs (KGs) provide structured representations of entities and relations, and underpin data management and Web applications such as search, recommendation, and question answering. Since Web-scale KGs such as Freebase and Wikidata are inherently incomplete, knowledge graph embedding (KGE) maps entities and relations into continuous vector spaces and is widely used to infer missing facts through link prediction. Despite extensive progress in KGE model design, the role of regularization has been less systematically examined, especially beyond tensor-decomposition models. The primary contribution of this paper is a comprehensive empirical study of regularization for KGE from the perspective of generalization. Across translation-based, tensor-decomposition-based, GNN-based, temporal, and multimodal KGE models, we show that overfitting is common across model families and that appropriate regularization can substantially improve both generalization and predictive performance. Motivated by these findings, we further propose Sparse Peak Regularization (SPR), a simple and model-agnostic regularizer that introduces rank-based selective sparsity into the regularization target. SPR concentrates the penalty on dominant high-energy embedding components, which contribute most to the embedding norm and overfitting risk, while leaving low-energy components unpenalized so that weak but useful signals are not unnecessarily suppressed. Experiments on eight benchmark datasets and diverse KGE models show that SPR achieves the best results in nearly all settings compared with existing regularizers such as N3, DURA, and ER. These results show that regularization deserves more systematic attention in KGE, and that selective regularization is an effective and broadly applicable strategy for improving the generalization of KGE models.
Abstract:Knowledge graphs (KGs) provide structured representations of entities and relations, and underpin data management and Web applications such as search, recommendation, and question answering. Since Web-scale KGs such as Freebase and Wikidata are inherently incomplete, knowledge graph embedding (KGE) maps entities and relations into continuous vector spaces and is widely used to infer missing facts through link prediction. Despite extensive progress in KGE model design, the role of regularization has been less systematically examined, especially beyond tensor-decomposition models. The primary contribution of this paper is a comprehensive empirical study of regularization for KGE from the perspective of generalization. Across translation-based, tensor-decomposition-based, GNN-based, temporal, and multimodal KGE models, we show that overfitting is common across model families and that appropriate regularization can substantially improve both generalization and predictive performance. Motivated by these findings, we further propose Sparse Peak Regularization (SPR), a simple and model-agnostic regularizer that introduces rank-based selective sparsity into the regularization target. SPR concentrates the penalty on dominant high-energy embedding components, which contribute most to the embedding norm and overfitting risk, while leaving low-energy components unpenalized so that weak but useful signals are not unnecessarily suppressed. Experiments on eight benchmark datasets and diverse KGE models show that SPR achieves the best results in nearly all settings compared with existing regularizers such as N3, DURA, and ER. These results show that regularization deserves more systematic attention in KGE, and that selective regularization is an effective and broadly applicable strategy for improving the generalization of KGE models.
Machine Learning (cs.LG); Computation and Language (cs.CL)
arXiv:2505.23442 [cs.LG]
(or arXiv:2505.23442v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2505.23442 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 342 cuvinte
Subiecte conexe
Grokking through the Lens of Minimum-Norm Interpolation
arXiv · 1 oct. 2026
Path Regularization: A Near-Complete and Optimal Nonasymptotic Generalization Theory for Multilayer Neural Networks and Double Descent Phenomenon
arXiv · 24 sept. 2026
Generalization and infinite width
LessWrong · 1 aug. 2026
Optimal Explicit and Implicit Regularization Strengths in High-Dimensional Continual Linear Regression
arXiv · 16 oct. 2026