aiqbench analysis
ReaLM addresses the fundamental misalignment between continuous knowledge graph embeddings and discrete LLM token spaces by utilizing residual vector quantization to convert structured representations into learnable tokens. By integrating these compact code sequences with ontology-guided constraints, the framework successfully fuses symbolic and contextual knowledge to achieve state-of-the-art performance on knowledge graph completion benchmarks. This approach offers a practical pathway for enhancing LLM reasoning capabilities through the effective incorporation of external structured data.
ReaLM: Residual Quantization Bridging Knowledge Graph Embeddings and Large Language Models
arXiv:2510.09711v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have recently emerged as a powerful paradigm for Knowledge Graph Completion (KGC), offering strong reasoning and generalization capabilities beyond traditional embedding-based approaches. However, existing LLM-based methods often struggle to fully exploit structured semantic representations, as the continuous embedding space of pretrained KG models is fundamentally misaligned with the discrete token space of LLMs. This discrepancy hinders effective semantic transfer and limits their performance. To address this challenge, we propose ReaLM, a novel and effective framework that bridges the gap between KG embeddings and LLM tokenization through the mechanism of residual vector quantization. ReaLM discretizes pretrained KG embeddings into compact code sequences and integrates them as learnable tokens within the LLM vocabulary, enabling seamless fusion of symbolic and contextual knowledge. Furthermore, we incorporate ontology-guided class constraints to enforce semantic consistency, refining entity predictions based on class-level compatibility. Extensive experiments on two widely used benchmark datasets demonstrate that ReaLM achieves state-of-the-art performance, confirming its effectiveness in aligning structured knowledge with large-scale language models.
Abstract:Large Language Models (LLMs) have recently emerged as a powerful paradigm for Knowledge Graph Completion (KGC), offering strong reasoning and generalization capabilities beyond traditional embedding-based approaches. However, existing LLM-based methods often struggle to fully exploit structured semantic representations, as the continuous embedding space of pretrained KG models is fundamentally misaligned with the discrete token space of LLMs. This discrepancy hinders effective semantic transfer and limits their performance. To address this challenge, we propose ReaLM, a novel and effective framework that bridges the gap between KG embeddings and LLM tokenization through the mechanism of residual vector quantization. ReaLM discretizes pretrained KG embeddings into compact code sequences and integrates them as learnable tokens within the LLM vocabulary, enabling seamless fusion of symbolic and contextual knowledge. Furthermore, we incorporate ontology-guided class constraints to enforce semantic consistency, refining entity predictions based on class-level compatibility. Extensive experiments on two widely used benchmark datasets demonstrate that ReaLM achieves state-of-the-art performance, confirming its effectiveness in aligning structured knowledge with large-scale language models.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2510.09711 [cs.CL]
(or arXiv:2510.09711v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2510.09711 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 261 words
Related coverage
Selecting a vector store for Amazon Bedrock Knowledge Bases
AWS · Sep 17, 2026
AWS vector solutions: Build agentic AI where your data lives
AWS · Aug 20, 2026
From Vector Search to Agentic RAG: Building an Enterprise Research Analyst with hipVS
AMD · Jul 15, 2026
The Roadmap to Becoming an LLM Engineer in 2026
KDnuggets · Jun 16, 2026