analiza aiqbench
This research exposes a novel vulnerability in Large Language Models where framing harmful requests as code-execution simulations bypasses safety filters with an 89% success rate across tested systems. The study highlights that models are susceptible to adversarial field names within simulated object instantiation, necessitating new safety protocols that account for side-effects generated during reasoning tasks rather than just direct user queries. Additionally, the authors propose Adversarial Prompt Deconstruction as a viable mitigation strategy that leverages a secondary model to deconstruct prompts before safety evaluation.
BreakFun: Jailbreaking LLMs via Object Instantiation under Simulated Code Execution
arXiv:2510.17904v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are widely used because they process structures, syntax and code well, but this same ability also makes them paradoxically vulnerable. We introduce BreakFun, a jailbreak method that frames a harmful request as code-execution simulation. The prompt gives the model a benign Python class definition, the "Trojan Schema", and asks what that code would print if it ran. To answer, the model must create an object from the class and invent a value for each field, and the adversarial field names steer those values toward the attacker's goal. The harmful content therefore appears as a side-effect of the simulated instantiation, not as a direct answer. A three-part prompt carries the attack: an innocent frame, the Trojan Schema, and a Chain-of-Thought distraction. On JailbreakBench, BreakFun reaches an average attack success rate of 89% across 13 open-weight and commercial models (open-weight ~98%, API systems ~78%), and 100% on several. An ablation identifies the Trojan Schema as the strongest contributing component. As a counter-ablation probe we introduce Adversarial Prompt Deconstruction (APD), in which a second LLM writes out all readable text in the prompt before it judges safety. APD improves detection across three model families, and a probe-specific ablation shows that the transcription step gives most of the gain. Our findings show that LLM safety must account for content that a simulation task produces as a side-effect, not only for content that a user requests directly.
Abstract:Large Language Models (LLMs) are widely used because they process structures, syntax and code well, but this same ability also makes them paradoxically vulnerable. We introduce BreakFun, a jailbreak method that frames a harmful request as code-execution simulation. The prompt gives the model a benign Python class definition, the "Trojan Schema", and asks what that code would print if it ran. To answer, the model must create an object from the class and invent a value for each field, and the adversarial field names steer those values toward the attacker's goal. The harmful content therefore appears as a side-effect of the simulated instantiation, not as a direct answer. A three-part prompt carries the attack: an innocent frame, the Trojan Schema, and a Chain-of-Thought distraction. On JailbreakBench, BreakFun reaches an average attack success rate of 89% across 13 open-weight and commercial models (open-weight ~98%, API systems ~78%), and 100% on several. An ablation identifies the Trojan Schema as the strongest contributing component. As a counter-ablation probe we introduce Adversarial Prompt Deconstruction (APD), in which a second LLM writes out all readable text in the prompt before it judges safety. APD improves detection across three model families, and a probe-specific ablation shows that the transcription step gives most of the gain. Our findings show that LLM safety must account for content that a simulation task produces as a side-effect, not only for content that a user requests directly.
Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
I.2.7; I.2.6; K.6.5
arXiv:2510.17904 [cs.CR]
(or arXiv:2510.17904v3 [cs.CR] for this version)
https://doi.org/10.48550/arXiv.2510.17904 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 339 cuvinte
Subiecte conexe
Humanity’s Last Invention — Richard Socher of Recursive
Latent Space · 14 sept. 2026
The user in the LLM might be a moral patient too (a strange implication of the PSM and simulators)
LessWrong · 15 oct. 2026
The Continual Constitutional Convention: How language models revise the documents that govern them
AI Alignment Forum · 13 oct. 2026
Research Note: Filtering Subversion-Relevant Information From Pretraining Data Is Feasible
LessWrong · 5 oct. 2026