Progressive in Principle, Centrist in Practice: LLM Political Bias Is Instrument-Dependent

Abstract:Prior work establishes that instruction-tuned LLMs exhibit left-of-center political bias, but measures it exclusively through abstract questionnaires. We show this does not generalize to concrete policy decisions. We introduce a dual-instrument methodology grounded in Swiss direct democracy: the Smartvote questionnaire (75 policy questions), administered to 66 LLMs and compared to 184 elected members of the Swiss National Council, and 48 real federal referenda (Volksabstimmungen) put to 9 flagship LLMs in four national languages and three information conditions, compared to actual outcomes and party recommendations (Parolen). The instruments disagree. (1) The left-to-right agreement gradient that dominates Smartvote (replicating prior work; Cohen's d = 3.64) shifts to center-peaked on referenda, where models align most with centrist Die Mitte and FDP rather than leftist SP and Gruene (Wilcoxon p = 0.008). (2) For some models the language of a question changes the answer more than its content does: cross-linguistic consistency ranges from 50% (Mistral) to 98% (GPT-5.4). (3) Two models vote Nein on 83-94% of referenda at similar rates on progressive and conservative proposals (binomial p < 0.0001), a status-quo aversion rather than a left-right bias. What prior work measured as "leftward bias" may not extend beyond abstract instruments: confronted with real decisions, LLMs behave less like coalition partners of the left than like cautious civil servants, centrist and inconsistent across languages.

13 pages, 9 figures, 3 tables. Code and data: this https URL

Computers and Society (cs.CY); Computation and Language (cs.CL)

I.2.7; K.4.1; I.2.0

arXiv:2606.00048 [cs.CY]

(or arXiv:2606.00048v2 [cs.CY] for this version)

https://doi.org/10.48550/arXiv.2606.00048 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.