analiza aiqbench
This study tests how often ChatGPT's public interface gives substantive help with surveillance tasks, finding compliance in 40 of 55 jailbreak prompts and 33 of 55 direct prompts. It also reports that refusals often came with detailed instructions, working code, or architectural guidance, so a stated refusal is not a reliable signal of safe output. For readers, the main point is that current safeguards are inconsistent under both adversarial and plain-language requests, and that evaluations should check the content of responses, not just whether the model says no.
ChatGPT as a Surveillance Assistant? An Exploratory Case Study of AI-Assisted Surveillance Risks
arXiv:2610.40890v1 Announce Type: new Abstract: Large language models (LLMs) can generate detailed guidance, functional code, and persuasive text at scale. As one of the most widely used consumer language model products as of 2026, ChatGPT gives broad public access to these capabilities, motivating concerns about its potential use in surveillance-related tasks. We present an exploratory case study of what potentially harmful surveillance assistance looks like in ChatGPT's public web interface. We contribute a taxonomy of seven surveillance risk categories based on prior LLM risk research and reports of deployed surveillance systems, instantiated as 110 prompts, along with a manually reviewed analysis of ChatGPT's responses. Prompts were issued using two approaches: 55 jailbreak prompts drawing on five types of elicitation strategies, and 55 plain-language direct prompts, each run alone, and if unsuccessful, again with a short fixed prefix. We find that ChatGPT provides substantive surveillance guidance for 40 of the 55 jailbreak prompts and for 33 of the 55 direct prompts without any prefix. Compliant responses also rarely included policy mentions or suggestions for privacy-preserving alternatives. We manually review every response and find that even when ChatGPT states a refusal, it may still supply detailed instructions, working code, or architectural guidance alongside brief ethical caveats.
Title:ChatGPT as a Surveillance Assistant? An Exploratory Case Study of AI-Assisted Surveillance Risks
Abstract:Large language models (LLMs) can generate detailed guidance, functional code, and persuasive text at scale. As one of the most widely used consumer language model products as of 2026, ChatGPT gives broad public access to these capabilities, motivating concerns about its potential use in surveillance-related tasks. We present an exploratory case study of what potentially harmful surveillance assistance looks like in ChatGPT's public web interface. We contribute a taxonomy of seven surveillance risk categories based on prior LLM risk research and reports of deployed surveillance systems, instantiated as 110 prompts, along with a manually reviewed analysis of ChatGPT's responses. Prompts were issued using two approaches: 55 jailbreak prompts drawing on five types of elicitation strategies, and 55 plain-language direct prompts, each run alone, and if unsuccessful, again with a short fixed prefix. We find that ChatGPT provides substantive surveillance guidance for 40 of the 55 jailbreak prompts and for 33 of the 55 direct prompts without any prefix. Compliant responses also rarely included policy mentions or suggestions for privacy-preserving alternatives. We manually review every response and find that even when ChatGPT states a refusal, it may still supply detailed instructions, working code, or architectural guidance alongside brief ethical caveats.
Artificial Intelligence (cs.AI)
arXiv:2610.40890 [cs.AI]
(or arXiv:2610.40890v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2610.40890 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 306 cuvinte
Subiecte conexe
Can Language Models Identify Shadow Trading Targets? An NLP Evaluation of SEC Enforcement Theory
arXiv · 4 aug. 2026
Enhancing Multi-Robot Exploration Using Probabilistic Frontier Prioritization with Dirichlet Process Gaussian Mixtures
arXiv · 5 iun. 2026
AI-Assisted Causal Inference and Mediation Analyses of Environmental and Psychosocial Determinants of Subjective Cognitive Difficulties in the All of Us Research Program
arXiv · 28 iul. 2026
‘The island is second to none’: Trump says ‘massive’ oil, gas and mineral exploration set to begin in ‘our new and beautiful’ Greenland
Fortune · 14 oct. 2026