aiqbench analysis
This preprint shows that black-box text injection can steer decision models toward attacker-chosen wrong answers, even when those answers are in the allowed set. Success rates reach 32 to 99% on initially correct decisions, and accuracy and confidence scores do not reliably flag the problem. For practitioners, restricting output format is not a robustness guarantee, so per-model, per-task, and per-version adversarial testing is needed. Results come from a single unreviewed arXiv preprint, so treat the specific figures as provisional.
JevBreak: Your Model, Their Decision
arXiv:2611.03759v1 Announce Type: cross Abstract: Restricting what a model can say is often mistaken for restricting an attacker's influence over its decision. An attacker can add text to the evidence and steer the model toward a wrong answer that still belongs to the allowed set. We evaluate seven decision models spanning hosted APIs, open-weight models and an encoder on 16 tasks in seven domains from Decision Bench, targeting every wrong option across 240 text examples (990 targets per model). We present Slot Attack, a black-box attack designed for decision models that appends text assembled from named components to steer predictions toward an attacker-selected incorrect option. The method relies solely on returned label probabilities, achieves success rates of 32--99\% on initially correct decisions, and identifies the effective component combinations with a median of 17--72 queries per successful attack. Accuracy and confidence do not reliably reveal this vulnerability. Models with near-identical accuracy on the sampled tasks differ by about 20\%age points in attack success. We demonstrate that an attacker can force not just one wrong answer but every available one: on about 20\% of correct decisions for the JEV models, 77\% for gpt-6-luna and 98\% for GLiNER2.5; a confidence threshold does not reliably screen the attack out. A fixed answer set constrains a decision's form, not its source; robustness must be measured per model, per task and per version rather than assumed from output format.
Abstract:Restricting what a model can say is often mistaken for restricting an attacker's influence over its decision. An attacker can add text to the evidence and steer the model toward a wrong answer that still belongs to the allowed set. We evaluate seven decision models spanning hosted APIs, open-weight models and an encoder on 16 tasks in seven domains from Decision Bench, targeting every wrong option across 240 text examples (990 targets per model). We present Slot Attack, a black-box attack designed for decision models that appends text assembled from named components to steer predictions toward an attacker-selected incorrect option. The method relies solely on returned label probabilities, achieves success rates of 32--99\% on initially correct decisions, and identifies the effective component combinations with a median of 17--72 queries per successful attack. Accuracy and confidence do not reliably reveal this vulnerability. Models with near-identical accuracy on the sampled tasks differ by about 20\%age points in attack success. We demonstrate that an attacker can force not just one wrong answer but every available one: on about 20\% of correct decisions for the JEV models, 77\% for gpt-6-luna and 98\% for GLiNER2.5; a confidence threshold does not reliably screen the attack out. A fixed answer set constrains a decision's form, not its source; robustness must be measured per model, per task and per version rather than assumed from output format.
Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
arXiv:2611.03759 [cs.CR]
(or arXiv:2611.03759v1 [cs.CR] for this version)
https://doi.org/10.48550/arXiv.2611.03759 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 326 words