aiqbench analysis
This research challenges standard evaluation methods for LLM bias by demonstrating that many reported sensitivities to specific factors, like patient gender, are actually artifacts of general text surface variation rather than true model behavior. The authors propose a rigorous statistical framework comparing targeted interventions against meaning-preserving paraphrases to isolate genuine effects from incidental noise. This work is critical for establishing robust baselines in AI safety and bias auditing, ensuring that future evaluations accurately distinguish between real semantic biases and superficial linguistic triggers.
Compared to What? Baselines and Metrics for Counterfactual Prompting
arXiv:2605.01048v2 Announce Type: replace-cross Abstract: Counterfactual prompting (i.e., perturbing a single factor and measuring output change) is widely used to evaluate things like LLM bias and CoT faithfulness. But in this work we argue that observed effects cannot be attributed to the targeted factor without accounting for baseline "meaning-preserving" modifications to text that establish general model sensitivity. This is because every counterfactual edit is a compound treatment that bundles the variable of interest with incidental surface-form variation; this violates treatment variation irrelevance. We observe prediction flip rates on MedQA of 14.9% when we surgically change patient gender. However, this is statistically indistinguishable from the flip rates induced by simply paraphrasing inputs (14.1%). In this case, it would therefore be unwarranted to conclude that the LLM is especially sensitive to patient gender. To account for this and robustly measure the effects of targeted interventions, we propose a framework in which we compare (via statistical testing) differences observed under target interventions to those induced by paraphrasing inputs. We then use this framework to revisit a analysis done on the MedPerturb dataset, which reported evidence of model sensitivity to patient demographics and stylistic cues. We find that these effects largely dissipate when we account for general model sensitivity, with only 5 of 120 tests reaching statistical significance. Applying the same framework to occupational biography classification, we detect clearly significant directional gender bias, showing that the framework identifies real directional effects even when they are small. We evaluate a range of metrics (aggregate, per-sample distributional, and regression) and find that per-sample metrics are dramatically more powerful than aggregate metrics and regression powerfully and uniquely characterizes effect direction and magnitude.
Title:Compared to What? Baselines and Metrics for Counterfactual Prompting
Abstract:Counterfactual prompting (i.e., perturbing a single factor and measuring output change) is widely used to evaluate things like LLM bias and CoT faithfulness. But in this work we argue that observed effects cannot be attributed to the targeted factor without accounting for baseline "meaning-preserving" modifications to text that establish general model sensitivity. This is because every counterfactual edit is a compound treatment that bundles the variable of interest with incidental surface-form variation; this violates treatment variation irrelevance. We observe prediction flip rates on MedQA of 14.9% when we surgically change patient gender. However, this is statistically indistinguishable from the flip rates induced by simply paraphrasing inputs (14.1%). In this case, it would therefore be unwarranted to conclude that the LLM is especially sensitive to patient gender. To account for this and robustly measure the effects of targeted interventions, we propose a framework in which we compare (via statistical testing) differences observed under target interventions to those induced by paraphrasing inputs. We then use this framework to revisit a analysis done on the MedPerturb dataset, which reported evidence of model sensitivity to patient demographics and stylistic cues. We find that these effects largely dissipate when we account for general model sensitivity, with only 5 of 120 tests reaching statistical significance. Applying the same framework to occupational biography classification, we detect clearly significant directional gender bias, showing that the framework identifies real directional effects even when they are small. We evaluate a range of metrics (aggregate, per-sample distributional, and regression) and find that per-sample metrics are dramatically more powerful than aggregate metrics and regression powerfully and uniquely characterizes effect direction and magnitude.
Published as a conference paper at COLM 2026. 33 pages, 10 figures, 18 tables. Code: this https URL ; Python package (cfprompt): this https URL
Computation and Language (cs.CL); Machine Learning (cs.LG)
arXiv:2605.01048 [cs.CL]
(or arXiv:2605.01048v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2605.01048 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 400 words
Related coverage
Why do models task game?
AI Alignment Forum · Aug 6, 2026
Untrusted advice for AI control: Short, strong advice significantly uplifts weak LLMs
LessWrong · Jul 27, 2026
Coherence-Aware Distributional Evaluation of Open-Ended Text Generation
arXiv · Sep 29, 2026
Informe de riesgos de la IA de frontera (febrero–marzo de 2026)
METR · May 19, 2026