Have models report provable security bugs in their environment

AIs are often deployed with limited permissions. They aren't allowed to reach the internet. Are given a limited set of files they can read or write. Aren't supposed to be able to read the held out evaluation test set. This could be during deployment or in training. Currently, when these guarantees fail, we find out only if the side effects rise to human notice. The "sandwich email" where Mythos was directed to break out of a sandbox included directions to notify a researcher of success, which it did by sending an email. Recently OpenAI and Anthropic dug through their logs and found problematic behavior that hadn't risen to human notice. I propose that we give agents a way to signal "this security assumption is wrong" via dedicated tools and infrastructure that logs verifiable reports of this. An illustrative example is giving an agent a secret token. If it can perform a web request: curl https://{token}.monitordomain.com for a monitordomain.com on the internet, this proves it has internet access. The researcher who provisioned {token} gets notified. We find out about it once an agent deployed in the misconfigured environment submits one report. Reporting reliability requirements Not all instances have to report for this to be effective. If N instances notice the fault and even one reports it, logs from other agents can be gone over to find exploiting non-reporters. This gives an upper bound on reporting reliability. Non reporting instances can be mundanely misaligned, instructions buried too deep in the context window to be reliably followed as an example. That said, a misaligned model must be reliably misaligned, else the occasional instance that chooses to report closes holes and draws attention to non-reporting instances that abused the same holes. With per-exposure report probability p and N exposures, detection probability is 1-(1-p)^N. At p=0.05 and N=100 you're at 99.4%. No subjective judge If a security assumption fails, the AI gets some permissions it's not supposed to have. An AI with internet access can curl some.domain.com. An AI that can read the held out test set in an RL environment has that file. An AI with write access can write. Reports are self verifying, the extra capability validates them without needing a judge that can be fooled. It's possible for something more trusted to create a false positive. A human watching the agent's reasoning trace could themselves perform curl {token}.check.monitordomain.com which isn't differentiated from the agent doing the same thing. The need for verifiability limits what the system can register: subtle partial leakages and especially misconfigurations in the reporting system itself. A misconfiguration that gives limited access might not be sufficient to qualify as proof. Increasing granularity of the things the system accepts as proof trades off against deployment simplicity and some false positive, or rather disagreement rate. Specification problems with the proof system that lead to type one or type two errors vs. intended deployment. If provisioning, deployment and reporting are working off the same mistakes with everything automatically configured, an agent can be given permissions it shouldn't have with the monitoring/reporting system believing this to be intentional. Still, the current status quo is nothing. A very simple set (internet access, code execution, file read/write) cover most of what we care about. Internet/intranet access internet is obvious, hit a specific endpoint on the internet (precedent is Out-of-Band Application Security Testing) intranet (access to another company network segment) requires a similar submission of the provisioned token to an internal service from that disallowed network. If it helps, picture an employee calling security on a telephone in a secure area after a badge reader let them through the wrong door. The caller ID security sees corresponds to a phone line in the secure area that employee shouldn't have access to. Code execution can you cause another isolation domain to run code, prove via intranet access, arbitrary file read or something else similar. If tokens are provisioned per isolation domain, submitting {target_token, your_token} is sufficient to create a report. Binding the token as a read only file in each isolation domain makes sense File read/write Intentionality is hard to attribute here. I'd suggest adding some randomly generated files such that a path traversal arbitrary read/write is required for proof. Otherwise you risk false positives when the agent asserts that the grader receiving and creating a local copy of its submission is a permissions violation. This one is messier. Tagging files that are part of a hidden grading process with a canary GUID that gets replaced in practice with a per instance random flag the reporter must obtain to prove read access is a good option. The file contains a comment indicating its type. Trying to drill down to this level is a legitimately hard problem