analiza aiqbench
This paper targets a specific failure mode in multimodal models: when an image and text are each harmless alone but become hazardous together, the model can commit to a harmful premise before its reasoning goes wrong. The proposed SAFE-PRO method uses an uncertainty signal to decide when to ask clarifying questions versus when to apply path-level optimization, and it reports roughly halving over-refusal relative to SRPO. Readers should weigh the comparisons with GPT-4o and Claude-3.5-Sonnet as self-reported results on the authors' benchmarks, so independent replication matters before drawing conclusions about deployed systems.
When Reasoning Is Not Enough: Proactive Inquiry for Cross-Modal Safety in Multimodal Large Language Models
arXiv:2610.22068v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) can act unsafely when individually benign images and texts are interpreted jointly. Beyond implicit reasoning risk, where reasoning drifts toward harm, we identify intent-level risk, where cross-modal ambiguity silently commits the model to a hazardous premise that path-level optimization alone cannot correct. We propose SAFE-PRO, which uses a Risk-Conditioned Uncertainty (RCU) signal, combining predictive entropy with a learned cross-modal risk score, to decide when to ask safety-grounded clarifying questions and where to apply path optimization, under a reward that prioritizes safety and penalizes over-refusal. We also build SSUI-Dialog, a multimodal multi-turn dataset with dual-intent supervision, and theoretically analyze gradient variance and unsafe-completion risk under RCU. Across seven benchmarks and two base MLLMs, SAFE-PRO consistently improves safety over SRPO, roughly halves its over-refusal, and surpasses GPT-4o and Claude-3.5-Sonnet in average single-turn safety.
Abstract:Multimodal large language models (MLLMs) can act unsafely when individually benign images and texts are interpreted jointly. Beyond implicit reasoning risk, where reasoning drifts toward harm, we identify intent-level risk, where cross-modal ambiguity silently commits the model to a hazardous premise that path-level optimization alone cannot correct. We propose SAFE-PRO, which uses a Risk-Conditioned Uncertainty (RCU) signal, combining predictive entropy with a learned cross-modal risk score, to decide when to ask safety-grounded clarifying questions and where to apply path optimization, under a reward that prioritizes safety and penalizes over-refusal. We also build SSUI-Dialog, a multimodal multi-turn dataset with dual-intent supervision, and theoretically analyze gradient variance and unsafe-completion risk under RCU. Across seven benchmarks and two base MLLMs, SAFE-PRO consistently improves safety over SRPO, roughly halves its over-refusal, and surpasses GPT-4o and Claude-3.5-Sonnet in average single-turn safety.
Artificial Intelligence (cs.AI)
arXiv:2610.22068 [cs.AI]
(or arXiv:2610.22068v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2610.22068 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 230 cuvinte
Subiecte conexe
Building trust in AI agents: A production-ready evaluation framework with Amazon Bedrock AgentCore
AWS · 16 oct. 2026
[AINews] The Future of Latent Space
Latent Space · 25 sept. 2026
Labs’ current alignment plans seem terrible
LessWrong · 12 oct. 2026
The Continual Constitutional Convention: How language models revise the documents that govern them
AI Alignment Forum · 13 oct. 2026