analiza aiqbench
This research introduces NarrativeAttack, a novel jailbreak framework that exploits the narrative reasoning capabilities of Unified Multimodal Models (UMMs) to bypass safety filters via visual context manipulation. By structuring attacks as three-act stories where generated images serve as reasoning steps, the method achieves high success rates against models like Gemini-2.5-Flash, revealing critical gaps in current alignment strategies for multimodal systems. The findings underscore an urgent need for enhanced safety protocols specifically designed to handle cross-turn visual grounding and narrative-based prompt injection in generative AI workflows.
The Plot Twist: Jailbreaking Unified Multimodal Models with a Three-Act NarrativeAttack
arXiv:2509.26473v2 Announce Type: replace Abstract: Unified Multimodal Understanding and Generation Models (UMMs) increasingly combine visual understanding and image generation within a single interactive workflow, making generated visual content available as later reasoning context. However, existing jailbreak evaluations mostly study text rewriting or isolated visual prompts, leaving the safety risk of narrative cross-turn visual grounding underexplored. We propose NarrativeAttack, a semantic-preserving visual narrative jailbreak framework. NarrativeAttack employs a three-act narrative structure in which the UMM's own generator produces images for the setup (pre-event) and resolution (post-event) stages, making the full attack workflow self-contained while concealing the malicious event as a hidden climax. The attack concludes with an image-based "guessing game" that embeds the original malicious query among benign candidates, compelling the model to select and answer the most relevant one based on the established narrative context. A dynamic difficulty mechanism further enhances attack stability. Experiments show NarrativeAttack consistently surpasses prior approaches, achieving up to 88.25% ASR on Gemini-2.5-Flash. These results uncover an underdeveloped vulnerability and highlight the urgent need for safety alignment in UMMs.
Abstract:Unified Multimodal Understanding and Generation Models (UMMs) increasingly combine visual understanding and image generation within a single interactive workflow, making generated visual content available as later reasoning context. However, existing jailbreak evaluations mostly study text rewriting or isolated visual prompts, leaving the safety risk of narrative cross-turn visual grounding underexplored. We propose NarrativeAttack, a semantic-preserving visual narrative jailbreak framework. NarrativeAttack employs a three-act narrative structure in which the UMM's own generator produces images for the setup (pre-event) and resolution (post-event) stages, making the full attack workflow self-contained while concealing the malicious event as a hidden climax. The attack concludes with an image-based "guessing game" that embeds the original malicious query among benign candidates, compelling the model to select and answer the most relevant one based on the established narrative context. A dynamic difficulty mechanism further enhances attack stability. Experiments show NarrativeAttack consistently surpasses prior approaches, achieving up to 88.25% ASR on Gemini-2.5-Flash. These results uncover an underdeveloped vulnerability and highlight the urgent need for safety alignment in UMMs.
Artificial Intelligence (cs.AI)
arXiv:2509.26473 [cs.AI]
(or arXiv:2509.26473v2 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2509.26473 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 260 cuvinte
Subiecte conexe
Nemotron 3.5 Content Safety: Customizable Multimodal Safety for Global Enterprise AI
Hugging Face · 4 iun. 2026
Presentation: Engineering AI for Creativity and Curiosity on Mobile
InfoQ · 21 iul. 2026
Automating coherent long-form video generation
Google Research · 24 sept. 2026
Debating RSI, the US-China Gap, and Jaggedness with JS Denain of Epoch AI
Interconnects · 22 sept. 2026