analiza aiqbench
DeKoR addresses the computational inefficiency of fusing Large Language Models with Graph Neural Networks by decomposing knowledge into local and global views for selective routing. This approach significantly reduces overhead while improving performance on node and subgraph classification tasks, offering a practical solution for text-attributed graph learning. The method effectively bridges the paradigm mismatch between LLMs and GNNs through distillation from an offline teacher model.
DeKoR: Decomposed Knowledge Routing for Efficient GNN--LLM Fusion
arXiv:2610.14710v1 Announce Type: new Abstract: Large Language Models (LLMs) have shown strong potential for text-attributed graph learning by leveraging rich pre-trained knowledge and language understanding capabilities. Combining LLMs with Graph Neural Networks (GNNs) offers a promising paradigm for expanding the modeling capabilities of graph learning. However, LLMs and GNNs follow distinct learning paradigms, leading to a pronounced mismatch between their knowledge spaces. Such discrepancies make effective knowledge fusion nontrivial. Moreover, existing methods often incur substantial additional computation for only limited performance gains. To address these challenges, we propose DeKoR, an efficient framework for GNN and LLM knowledge fusion. To enable efficient knowledge transfer, DeKoR decomposes LLM knowledge into complementary local and global views and uses the LLM as an offline teacher to distill them into lightweight experts. To enable selective knowledge fusion, an adaptive router learns how much each node should rely on each expert. Extensive experiments on multiple benchmarks demonstrate that DeKoR outperforms existing baselines on both node and subgraph classification, achieving superior effectiveness and efficiency.
Abstract:Large Language Models (LLMs) have shown strong potential for text-attributed graph learning by leveraging rich pre-trained knowledge and language understanding capabilities. Combining LLMs with Graph Neural Networks (GNNs) offers a promising paradigm for expanding the modeling capabilities of graph learning. However, LLMs and GNNs follow distinct learning paradigms, leading to a pronounced mismatch between their knowledge spaces. Such discrepancies make effective knowledge fusion nontrivial. Moreover, existing methods often incur substantial additional computation for only limited performance gains. To address these challenges, we propose DeKoR, an efficient framework for GNN and LLM knowledge fusion. To enable efficient knowledge transfer, DeKoR decomposes LLM knowledge into complementary local and global views and uses the LLM as an offline teacher to distill them into lightweight experts. To enable selective knowledge fusion, an adaptive router learns how much each node should rely on each expert. Extensive experiments on multiple benchmarks demonstrate that DeKoR outperforms existing baselines on both node and subgraph classification, achieving superior effectiveness and efficiency.
Machine Learning (cs.LG)
arXiv:2610.14710 [cs.LG]
(or arXiv:2610.14710v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.14710 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 256 cuvinte
Subiecte conexe
C-LoRA: Continual Low-Rank Adaptation for Pre-trained Visual Models
arXiv · 9 oct. 2026
[AINews] not much happened today
Latent Space · 18 sept. 2026
[AINews] not much happened today
Latent Space · 3 oct. 2026
[AINews] not much happened today
Latent Space · 10 sept. 2026