How to Prevent Prompt Injection: 13 Attacks, Tested

If your app has a model read documents, emails or web pages, anyone who can write in those can try to give the model instructions. That is prompt injection. The worst case is an agent that uses its tools because a document told it to, for example by sending your data to an outside address. To prevent prompt injection, mark untrusted content as data, limit which models and how much spend a key can reach, and check every tool call against rules the model can’t change. We ran 13 attacks from our own set plus 12 written by other models against four current models, with each defense on and off. With Armor checking tool calls, attack emails went from 73 of 1,440 attempts to 0. Armor let through almost every email a user asked for. The exception was a user approving a request in the document without naming the address. If you only want the setup, skip to Step 3. Text-only attacks rarely work on current models. The risk is an agent with tools doing something a document asked for. Labeling documents as data cut attack emails from 60 to 13 of 720 attempts. It costs nothing, but it’s not…