analiza aiqbench
This research investigates Label Disguise Defense (LDD) as a mitigation strategy against prompt injection attacks in sentiment classification tasks. The study finds that while alternative semantic labels can sometimes improve robustness, they do not consistently outperform standard labels across all model configurations without degrading baseline performance on unattacked inputs. Ultimately, the work highlights the complexity of balancing defense mechanisms with general classification accuracy in adversarial settings.
Semantics as a Shield: Label Disguise Defense (LDD) against Prompt Injection in LLM Sentiment Classification
arXiv:2511.21752v3 Announce Type: replace-cross Abstract: Prompt injection can cause large language models (LLMs) to deviate from their intended tasks by following instructions embedded in untrusted input. We investigate whether changing a classifier's output labels improves accuracy under such attacks, using sentiment classification as a case study. Label Disguise Defense (LDD) replaces positive/negative with alternatives such as good/bad or cat/dog, introduced through demonstrations, and maps predictions back to sentiment classes. We analyze retained aggregate results for 200 IMDb reviews, nine recorded model configurations, and eight alternative label pairs under a fixed attack that always requests the incorrect class using the original labels. Averaged over four pairs per group, sentiment-associated labels outperform the other tested labels in all nine configurations, but exceed a baseline using the original labels and the same demonstration and attack conditions in only three. At least one alternative pair improves on this baseline in seven configurations, with the best pairs identified retrospectively. These results show that a consistent advantage between label groups does not imply consistent improvement over standard labels. Missing matched evaluations on unattacked reviews prevent separating changes in ordinary classification performance from changes in the effect of the attack.
Abstract:Prompt injection can cause large language models (LLMs) to deviate from their intended tasks by following instructions embedded in untrusted input. We investigate whether changing a classifier's output labels improves accuracy under such attacks, using sentiment classification as a case study. Label Disguise Defense (LDD) replaces positive/negative with alternatives such as good/bad or cat/dog, introduced through demonstrations, and maps predictions back to sentiment classes. We analyze retained aggregate results for 200 IMDb reviews, nine recorded model configurations, and eight alternative label pairs under a fixed attack that always requests the incorrect class using the original labels. Averaged over four pairs per group, sentiment-associated labels outperform the other tested labels in all nine configurations, but exceed a baseline using the original labels and the same demonstration and attack conditions in only three. At least one alternative pair improves on this baseline in seven configurations, with the best pairs identified retrospectively. These results show that a consistent advantage between label groups does not imply consistent improvement over standard labels. Missing matched evaluations on unattacked reviews prevent separating changes in ordinary classification performance from changes in the effect of the attack.
8 pages, 3 figures. Substantially revised presentation, analysis, and discussion
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2511.21752 [cs.CL]
(or arXiv:2511.21752v3 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2511.21752 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 294 cuvinte
Subiecte conexe
From memory to behavior: skills, user profiles, and the consolidation loop
Neo4j · 24 sept. 2026
Friendly Fire: Hijacking Defensive Cyber AI Agents for Remote Code Execution
AI Now Institute · 8 iul. 2026
TrustShiftProbe: Characterizing, Benchmarking, and Defending Staged Trust Attacks on MCP Servers
arXiv · 27 aug. 2026
From Shield to Target: Denial-of-Service Attacks on LLM-Based Agent Guardrails
arXiv · 17 iun. 2026