analiza aiqbench
This study demonstrates that explicitly prompting language models with pragmatic theories, such as Gricean pragmatics, significantly improves their ability to interpret implied meanings compared to standard chain-of-thought baselines. The approach yields up to a 9.6% performance gain on reasoning tasks, while even merely referencing these theories in the prompt provides a modest boost in larger models. For developers focusing on nuanced conversational AI, integrating theoretical frameworks into in-context learning strategies offers a viable path to enhancing interpretive accuracy without requiring extensive fine-tuning.
Pragmatic Theories Enhance Understanding of Implied Meanings in LLMs
arXiv:2510.26253v3 Announce Type: replace Abstract: The ability to accurately interpret implied meanings plays a crucial role in human communication and language use, and language models are also expected to possess this capability. This study demonstrates that providing language models with pragmatic theories as prompts is an effective in-context learning approach for tasks to understand implied meanings. Specifically, we propose an approach in which an overview of pragmatic theories, such as Gricean pragmatics and Relevance Theory, is presented as a prompt to the language model, guiding it through a step-by-step reasoning process to derive a final interpretation. Experimental results showed that, compared to the baseline, which prompts intermediate reasoning without presenting pragmatic theories (0-shot Chain-of-Thought), our methods enabled language models to achieve up to 9.6\% higher scores on pragmatic reasoning tasks. Furthermore, we show that even without explaining the details of pragmatic theories, merely mentioning their names in the prompt leads to a certain performance improvement (around 1-3%) in larger models compared to the baseline.
Abstract:The ability to accurately interpret implied meanings plays a crucial role in human communication and language use, and language models are also expected to possess this capability. This study demonstrates that providing language models with pragmatic theories as prompts is an effective in-context learning approach for tasks to understand implied meanings. Specifically, we propose an approach in which an overview of pragmatic theories, such as Gricean pragmatics and Relevance Theory, is presented as a prompt to the language model, guiding it through a step-by-step reasoning process to derive a final interpretation. Experimental results showed that, compared to the baseline, which prompts intermediate reasoning without presenting pragmatic theories (0-shot Chain-of-Thought), our methods enabled language models to achieve up to 9.6\% higher scores on pragmatic reasoning tasks. Furthermore, we show that even without explaining the details of pragmatic theories, merely mentioning their names in the prompt leads to a certain performance improvement (around 1-3%) in larger models compared to the baseline.
Computation and Language (cs.CL)
arXiv:2510.26253 [cs.CL]
(or arXiv:2510.26253v3 [cs.CL] for this version)
Text integral sub licența deschisă a publicației · 171 cuvinte
Subiecte conexe
Beyond the Literal: Decomposing Pragmatic Intent in Multimodal Meme Understanding
arXiv · 3 iun. 2026
Read the Room, Read the Image: Understanding Indirect Speech Acts in Multimodal Visual Contexts
arXiv · 1 sept. 2026
VakyArth: Evaluating Pragmatic Competence in LLMs across Indic Languages
arXiv · 3 sept. 2026
AGI Safety and Alignment at Google DeepMind: A Summary of Recent Work (July 2026)
AI Alignment Forum · 31 iul. 2026