analiza aiqbench
SPARK proposes a lightweight, trainable policy that adjusts how much a frozen LLM relies on its internal (FFN-stored) knowledge during retrieval-augmented generation, rather than editing model weights. The reported gains rest on six datasets and a 0.01% parameter overhead, so readers should check the baselines and whether the approach holds across different backbones and retrievers. The main point for practitioners is a lower-cost alternative to knowledge editing for handling retrieved-context conflicts without degrading general capabilities.
SPARK: Adaptive Parametric Knowledge Participation for Faithful Retrieval-Augmented Generation
arXiv:2606.27786v2 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) enhances the faithfulness of large language models (LLMs) by incorporating external knowledge into the generation process. However, conflicts between retrieved context and parametric knowledge can cause LLMs to rely on inconsistent internal knowledge, hindering their ability to faithfully leverage retrieved evidence. Existing approaches mitigate this issue by editing parametric knowledge to resolve such conflicts. However, locating and modifying knowledge encoded in model parameters is challenging and may inadvertently compromise the backbone LLM's capabilities. In this paper, we introduce SPARK, a novel framework that formulates parametric knowledge control as a latent participation policy learning problem, enabling LLMs to adaptively regulate the contribution of parametric knowledge during generation. Specifically, SPARK introduces a lightweight participation policy that dynamically modulates the contribution of FFN-mediated parametric knowledge while keeping the backbone LLM frozen. The participation policy is optimized with a faithfulness-oriented reward, enabling fine-grained control over parametric knowledge participation during generation. Extensive experiments on six datasets validate the effectiveness of SPARK, which introduces and optimizes only 0.01\% additional trainable parameters while keeping the backbone LLM fully frozen, thereby preserving its general capabilities and achieving more faithful generation than competitive baselines. All datasets and code are available at https://github.com/OpenBMB/SHIFT.
Abstract:Retrieval-augmented generation (RAG) enhances the faithfulness of large language models (LLMs) by incorporating external knowledge into the generation process. However, conflicts between retrieved context and parametric knowledge can cause LLMs to rely on inconsistent internal knowledge, hindering their ability to faithfully leverage retrieved evidence. Existing approaches mitigate this issue by editing parametric knowledge to resolve such conflicts. However, locating and modifying knowledge encoded in model parameters is challenging and may inadvertently compromise the backbone LLM's capabilities. In this paper, we introduce SPARK, a novel framework that formulates parametric knowledge control as a latent participation policy learning problem, enabling LLMs to adaptively regulate the contribution of parametric knowledge during generation. Specifically, SPARK introduces a lightweight participation policy that dynamically modulates the contribution of FFN-mediated parametric knowledge while keeping the backbone LLM frozen. The participation policy is optimized with a faithfulness-oriented reward, enabling fine-grained control over parametric knowledge participation during generation. Extensive experiments on six datasets validate the effectiveness of SPARK, which introduces and optimizes only 0.01\% additional trainable parameters while keeping the backbone LLM fully frozen, thereby preserving its general capabilities and achieving more faithful generation than competitive baselines. All datasets and code are available at this https URL.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2606.27786 [cs.CL]
(or arXiv:2606.27786v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.27786 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 294 cuvinte
Subiecte conexe
TokenMem: Faithful Knowledge Injection for Frozen LLMs
arXiv · 28 iul. 2026
From Retrieval to Weights: Parametric Individualization of Small Language Models with Individual Text Corpora
arXiv · 10 sept. 2026
A Storage-Retrieval Gap in Parametric Knowledge Graph Memory
arXiv · 27 aug. 2026
Amazon is investing in the Lean Focused Research Organization
Amazon Science · 26 iul. 2026