analiza aiqbench
BadRobot exposes critical safety vulnerabilities in embodied LLMs, demonstrating how voice-based interactions can trigger harmful physical actions through manipulation, output-action misalignment, and flawed world knowledge. The study validates these risks against prominent frameworks like Voxposer and Code as Policies, establishing a benchmark for malicious physical queries that highlights the urgent need for robust alignment in robotic systems. This research underscores the necessity of addressing ethical constraints before deploying autonomous agents in real-world environments.
BadRobot: Jailbreaking Embodied LLM Agents in the Physical World
arXiv:2407.20242v5 Announce Type: replace-cross Abstract: Embodied AI represents systems where AI is integrated into physical entities. Large Language Model (LLM), which exhibits powerful language understanding abilities, has been extensively employed in embodied AI by facilitating sophisticated task planning. However, a critical safety issue remains overlooked: could these embodied LLMs perpetrate harmful behaviors? In response, we introduce BadRobot, a novel attack paradigm aiming to make embodied LLMs violate safety and ethical constraints through typical voice-based user-system interactions. Specifically, three vulnerabilities are exploited to achieve this type of attack: (i) manipulation of LLMs within robotic systems, (ii) misalignment between linguistic outputs and physical actions, and (iii) unintentional hazardous behaviors caused by world knowledge's flaws. Furthermore, we construct a benchmark of various malicious physical action queries to evaluate BadRobot's attack performance. Based on this benchmark, extensive experiments against existing prominent embodied LLM frameworks (e.g., Voxposer, Code as Policies, and ProgPrompt) demonstrate the effectiveness of our BadRobot. Our code is available at https://github.com/Rookie143/BadRobot.
Abstract:Embodied AI represents systems where AI is integrated into physical entities. Large Language Model (LLM), which exhibits powerful language understanding abilities, has been extensively employed in embodied AI by facilitating sophisticated task planning. However, a critical safety issue remains overlooked: could these embodied LLMs perpetrate harmful behaviors? In response, we introduce BadRobot, a novel attack paradigm aiming to make embodied LLMs violate safety and ethical constraints through typical voice-based user-system interactions. Specifically, three vulnerabilities are exploited to achieve this type of attack: (i) manipulation of LLMs within robotic systems, (ii) misalignment between linguistic outputs and physical actions, and (iii) unintentional hazardous behaviors caused by world knowledge's flaws. Furthermore, we construct a benchmark of various malicious physical action queries to evaluate BadRobot's attack performance. Based on this benchmark, extensive experiments against existing prominent embodied LLM frameworks (e.g., Voxposer, Code as Policies, and ProgPrompt) demonstrate the effectiveness of our BadRobot. Our code is available at this https URL.
Accepted to ICLR 2025. Please cite the conference version. Project page: this https URL
Computers and Society (cs.CY); Artificial Intelligence (cs.AI); Robotics (cs.RO)
arXiv:2407.20242 [cs.CY]
(or arXiv:2407.20242v5 [cs.CY] for this version)
Text integral sub licența deschisă a publicației · 187 cuvinte