aiqbench analysis
This research identifies a critical failure in multi-turn dialogue systems where models conflate rejected user proposals with active instructions, leading to persistent accuracy degradation. The authors introduce Intent-Eval, a new benchmark for tool actions and code generation, alongside Intent-OPSD, a self-distillation framework designed to train models on distinguishing between mentioned content and actual user intent. This work directly addresses the robustness of large language models in handling evolving conversational contexts without relying on external hype.
You Changed Your Mind, The Model Didn't: Demystifying Intent in Multi-Turn Dialogue
arXiv:2610.06496v1 Announce Type: new Abstract: When a large language model handles a multi-turn task and a user proposes a change but ultimately rejects it, the model should continue as if nothing changed. We find a surprising failure: merely mentioning a rejected change can derail task execution, even when the user's final intent remains unchanged. To systematically study language model behavior under evolving user intent, we introduce Intent-Eval, a controlled benchmark spanning tool actions, code, databases, and mathematics. Across diverse tasks, models are vulnerable to both rejected proposals and superseded requirements, consistent with mentioned-as-in-effect confusion: conversational content is treated as active requirements even after it has been rejected or replaced. Accuracy degradation can deepen or persist as interaction continues, highlighting the need to distinguish what has been mentioned from what remains in effect. Building on this insight, we propose Intent-OPSD, a decision-conditioned on-policy self-distillation framework with Teacher and Student initialized from the same model. The frozen Teacher provides active-intent supervision from the complete task matching the user's decision, training the Student on the full dialogue to follow active requirements reflecting user intent.
Abstract:When a large language model handles a multi-turn task and a user proposes a change but ultimately rejects it, the model should continue as if nothing changed. We find a surprising failure: merely mentioning a rejected change can derail task execution, even when the user's final intent remains unchanged. To systematically study language model behavior under evolving user intent, we introduce Intent-Eval, a controlled benchmark spanning tool actions, code, databases, and mathematics. Across diverse tasks, models are vulnerable to both rejected proposals and superseded requirements, consistent with mentioned-as-in-effect confusion: conversational content is treated as active requirements even after it has been rejected or replaced. Accuracy degradation can deepen or persist as interaction continues, highlighting the need to distinguish what has been mentioned from what remains in effect. Building on this insight, we propose Intent-OPSD, a decision-conditioned on-policy self-distillation framework with Teacher and Student initialized from the same model. The frozen Teacher provides active-intent supervision from the complete task matching the user's decision, training the Student on the full dialogue to follow active requirements reflecting user intent.
Artificial Intelligence (cs.AI)
arXiv:2610.06496 [cs.AI]
(or arXiv:2610.06496v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2610.06496 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 270 words
Related coverage
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · Sep 25, 2026
Presentation: From AI Agent Demo to Production: Automated Testing and Evaluation
InfoQ · Sep 7, 2026
Links #5: 2026/07
LessWrong · Jul 31, 2026
Open Distillation of Hereditary Traits
AI Alignment Forum · Jul 14, 2026