analiza aiqbench
This research introduces SCoNE, a training-free method that enhances Large Language Model robustness against retrieval noise by selectively editing Feed-Forward Network neurons. By strengthening context-aware pathways without fine-tuning or inference overhead, the approach significantly reduces hallucinations in Retrieval-Augmented Generation systems. This advancement directly addresses critical stability challenges in deploying AI models with external knowledge bases.
SCoNE: Selective Context-aware Neuron Editing for Robust Retrieval-Augmented Generation
arXiv:2609.00689v1 Announce Type: new Abstract: Retrieval-Augmented Generation (RAG) is highly sensitive to retrieval noise: when retrieved documents mix informative and irrelevant context, LLMs are easily distracted, leading to hallucinations. To overcome this, we propose SCoNE (Selective Context-aware Neuron Editing), a training-free model editing approach that improves retrieval noise robustness by selectively strengthening context-aware FFN neurons that are identified by both high attribution and high cross-input variability. SCoNE requires only a small number of mining samples, no fine-tuning, and no inference-time overhead. Across various knowledge-intensive question-answering benchmarks and two LLM backbones, SCoNE consistently outperforms competitive baseline methods. Our code is available at https://github.com/HYU-ARK-Lab/SCoNE.
Abstract:Retrieval-Augmented Generation (RAG) is highly sensitive to retrieval noise: when retrieved documents mix informative and irrelevant context, LLMs are easily distracted, leading to hallucinations. To overcome this, we propose SCoNE (Selective Context-aware Neuron Editing), a training-free model editing approach that improves retrieval noise robustness by selectively strengthening context-aware FFN neurons that are identified by both high attribution and high cross-input variability. SCoNE requires only a small number of mining samples, no fine-tuning, and no inference-time overhead. Across various knowledge-intensive question-answering benchmarks and two LLM backbones, SCoNE consistently outperforms competitive baseline methods. Our code is available at this https URL.
Computation and Language (cs.CL)
arXiv:2609.00689 [cs.CL]
(or arXiv:2609.00689v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.00689 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 194 cuvinte
Subiecte conexe
Where Does Robustness Live? Neuron-Guided Adaptation for Retrieval-Augmented Language Models
arXiv · 1 sept. 2026
[AINews] not much happened today
Latent Space · 10 sept. 2026
Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident
METR · 26 aug. 2026
Minimizing Targeted Activations: Input-Only Suppression of Evaluation-Awareness Latents in Large Language Models
arXiv · 29 iul. 2026