aiqbench analysis
This research introduces StakeBench, a new benchmark that shifts security testing from purely technical feasibility to an analysis of asymmetric harms across different stakeholders like consumers and merchants. By evaluating 264 adversarial cases in real-world shopping scenarios, the study reveals that current LLM-based web agents fail to resist any specific prompt-injection objective reliably. This work is critical for deploying safe AI agents where financial consequences are directly tied to user interactions with untrusted content.
Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agents
arXiv:2606.13385v2 Announce Type: replace-cross Abstract: LLM-based web agents are increasingly deployed in real-world settings such as e-commerce, where they interact extensively with untrusted web content while executing actions that carry direct financial consequences. This makes them vulnerable to prompt-injection attacks, in which seemingly benign web content conceals adversarial instructions that manipulate the agent's behavior. Existing security benchmarks adopt an \textit{attack-centric} perspective, focusing on the technical feasibility of injections while overlooking the nuanced distribution of resulting harms. In practice, however, prompt-injection risk is victim-dependent: a single exploit can produce asymmetric consequences for different stakeholders, and the same attack pattern may exhibit substantially different effectiveness depending on whom it targets. To capture these properties, we introduce StakeBench, a stakeholder-centric benchmark that systematically categorizes and attributes harm in real-world web agent systems for online shopping. In general, StakeBench decomposes prompt-injection risk into 12 concrete attack objectives across three stakeholder classes, realized by 22 reusable templates and instantiated into 264 executable adversarial cases spanning 12 product categories, with each case evaluated along complementary outcome- and process-level metrics. Evaluating four deployable agent-backbone configurations across 3,168 attacked runs, we find substantial and heterogeneous vulnerabilities: no attack objective is reliably resisted by current LLM-based web agents, and outcomes span four qualitatively distinct modes. These patterns are missed by conventional attack-centric, single-metric evaluation, underscoring the need for stakeholder-aware assessment of LLM-based agents in real-world deployments.
Authors:Zihao Wang, Yiming Li, Yutong Wu, Kangjie Chen, Zheyu Liu, Fok Kar Wai, Pin-Yu Chen, Vrizlynn L. L. Thing, Bo Li, Dacheng Tao, Tianwei Zhang
Abstract:LLM-based web agents are increasingly deployed in real-world settings such as e-commerce, where they interact extensively with untrusted web content while executing actions that carry direct financial consequences. This makes them vulnerable to prompt-injection attacks, in which seemingly benign web content conceals adversarial instructions that manipulate the agent's behavior. Existing security benchmarks adopt an \textit{attack-centric} perspective, focusing on the technical feasibility of injections while overlooking the nuanced distribution of resulting harms. In practice, however, prompt-injection risk is victim-dependent: a single exploit can produce asymmetric consequences for different stakeholders, and the same attack pattern may exhibit substantially different effectiveness depending on whom it targets. To capture these properties, we introduce StakeBench, a stakeholder-centric benchmark that systematically categorizes and attributes harm in real-world web agent systems for online shopping. In general, StakeBench decomposes prompt-injection risk into 12 concrete attack objectives across three stakeholder classes, realized by 22 reusable templates and instantiated into 264 executable adversarial cases spanning 12 product categories, with each case evaluated along complementary outcome- and process-level metrics. Evaluating four deployable agent-backbone configurations across 3,168 attacked runs, we find substantial and heterogeneous vulnerabilities: no attack objective is reliably resisted by current LLM-based web agents, and outcomes span four qualitatively distinct modes. These patterns are missed by conventional attack-centric, single-metric evaluation, underscoring the need for stakeholder-aware assessment of LLM-based agents in real-world deployments.
Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Computers and Society (cs.CY); Human-Computer Interaction (cs.HC); Multimedia (cs.MM)
arXiv:2606.13385 [cs.CR]
(or arXiv:2606.13385v2 [cs.CR] for this version)
Full text under the publisher's open licence · 271 words
Related coverage
Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agents
arXiv · Jun 12, 2026
Links #5: 2026/07
LessWrong · Jul 31, 2026
Honest Abacus AI Review: ChatLLM, DeepAgent, AI Studio & More
KDnuggets · Aug 4, 2026
OpenRouter: from Seed to Stripe — with OpenRouter’s Alex Atallah & AMP’s Anjney Midha
Latent Space · Sep 25, 2026