14 Human Oversight Patterns for AI Systems

We studied AI regulations, a variety of our customers, our internal systems, and published applications to identify human-in-the-loop patterns. We found 206 human controls across 225 AI regulations, and almost all of them fit within one of 14 patterns. A "human pattern" is a set of values for who decides, the deadline, the outcome when nobody decides, the allowed decisions and the record kept. Every AI system we have helped build reaches a point where a person has to decide. A payments agent matches an invoice to a purchase order and asks to pay the supplier 4,200 EUR. A notices agent drafts an adverse decision letter to a loan applicant. A model team proposes sending a new model 10% of production traffic. The payment, the letter and the rollout wait for a person to decide before the work continues. Our customers build each human decision with the Akka SDK. A workflow step pauses for the person and resumes when the decision arrives. A timeout on the pause can send a reminder or move the decision to someone more senior. An autonomous agent can hand a task to a person and wait for the result. The workflow and the task are…