analiza aiqbench
This research investigates prompt echoing in small instruction models, determining whether the behavior stems from training data leakage or misaligned internal induction mechanisms. The study concludes that while partial dataset overlap exists, the phenomenon is primarily driven by the model's copying architecture rather than simple memorization. This analysis offers critical insights into the reliability and failure modes of emerging lightweight AI architectures.
Mirror, Mirror on the Wall: Prompt Echoing in Small Instruct Language Models
arXiv:2609.15045v1 Announce Type: cross Abstract: Prompt echoing is a recognized failure mode of instruct language models, in which a model instead of generating a response, mirrors the provided prompt, even though it did not receive a specific instruction to do so. Is this phenomenon a sign of the model leaking the content of its training dataset, or is it rather caused by a misaligned behavior of the internal induction/copying mechanisms? We investigate prompt echoing small language models from different families (Gemma, Llama, Qwen, SmolLM and OLMo) and show that echoing prompts are likely to have partial overlap with the training dataset but the phenomenon is primarily driven by the model's induction heads.
Abstract:Prompt echoing is a recognized failure mode of instruct language models, in which a model instead of generating a response, mirrors the provided prompt, even though it did not receive a specific instruction to do so. Is this phenomenon a sign of the model leaking the content of its training dataset, or is it rather caused by a misaligned behavior of the internal induction/copying mechanisms? We investigate prompt echoing small language models from different families (Gemma, Llama, Qwen, SmolLM and OLMo) and show that echoing prompts are likely to have partial overlap with the training dataset but the phenomenon is primarily driven by the model's induction heads.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2609.15045 [cs.CL]
(or arXiv:2609.15045v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.15045 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 205 cuvinte
