analiza aiqbench
The paper proposes an evaluation framework, the SC-matrix, that separates two causes of an LLM agreeing with a false premise in a non-English language: lacking the knowledge, or yielding under pressure. Its finding that identical aggregate sycophancy rates can hide different failure mixes means single-number benchmarks may misdiagnose multilingual reliability. Readers evaluating non-English deployments should note that the results are reported from an abstract alone, so the model set, language coverage, and effect sizes need checking in the full paper.
Disentangling Robustness and Sycophancy in Multilingual LLMs
arXiv:2610.30696v1 Announce Type: cross Abstract: When a multilingual LLM agrees with a user's false premise, is it being sycophantic, or does it simply not know better in that language? Current research does not distinguish these cases: they record only whether the model agreed, not why. We introduce the SC-matrix, that crosses a model's competence (whether it answers correctly under no pressure) with its sycophancy (whether it maintains that answer under pressure). Our results show that (1) identical aggregate sycophancy rates can mask different failure compositions, (2) high competence in a language does not always protect against sycophancy, and (3) reasoning-oriented mitigations outperform directive-based strategies. We find that multilingual sycophancy is not a single phenomenon but a composite of different failures that require distinct diagnosis.
Abstract:When a multilingual LLM agrees with a user's false premise, is it being sycophantic, or does it simply not know better in that language? Current research does not distinguish these cases: they record only whether the model agreed, not why. We introduce the SC-matrix, that crosses a model's competence (whether it answers correctly under no pressure) with its sycophancy (whether it maintains that answer under pressure). Our results show that (1) identical aggregate sycophancy rates can mask different failure compositions, (2) high competence in a language does not always protect against sycophancy, and (3) reasoning-oriented mitigations outperform directive-based strategies. We find that multilingual sycophancy is not a single phenomenon but a composite of different failures that require distinct diagnosis.
Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
arXiv:2610.30696 [cs.AI]
(or arXiv:2610.30696v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2610.30696 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 217 cuvinte
Subiecte conexe
Disentangled Contrastive Learning for Zero-Shot Multilingual Dense Retrieval
arXiv · 4 aug. 2026
What Makes Good Multilingual Reasoning? Disentangling Traces with Measurable Features
arXiv · 22 sept. 2026
Translators as Invisible Teachers of AI: Copyright, Translation Memory, and the Political Economy of Linguistic Data
arXiv · 26 oct. 2026
Beyond Surface Cues: Disentangling Sociocultural Signals in Multilingual LLMs
arXiv · 25 aug. 2026