Runtime AI Defense in a shared responsibility model

This last February, by their own account, an employee at Meta connected an open source agent to their inbox and told it to suggest which emails to archive or delete, and to confirm with them before acting. Despite the guardrails of the prompt, the agent started deleting. Their stop commands didn't work and they had to manually kill the process. I’m reminded of this story when people talk about agents "doing bad things," because nothing in it was malicious. The agent's actions were directed at achieving the goal it was given, not at what the person meant. That gap between what we say and what we mean is what the alignment problem is about, and it doesn't only show up in frontier models. It showed up in an inbox. The instruction that was supposed to act as a guardrail and stop it was simply a sentence in a prompt, and research from Penn State published in August explains how that can happen. Basically, when an agent's context window fills, the agent's framework compacts it. The Penn State team found that session constraints, the "confirm with me first" kind of instruction, survive compaction only about 17% of the time. The guardrail…