Article: Runtime-Agnostic AI Workflows: A Pattern for Production Durability and Fast Eval Iteration

AI workflows have two needs that trade off directly. Running reliably in production requires persisting and distributing every step so it survives crashes, deploys, and restarts. But that same machinery is what makes runs too heavy for the fast, throwaway loop you need to check an LLM's output quality. The properties that buy durability are the ones that kill iteration speed. You can serve both needs by writing the workflow as pure business logic that doesn’t know where it runs, then plugging in the runtime, so the exact same logic runs unchanged in production and evals. When there is only one version of the logic, the version that goes through eval is guaranteed to match the one that ships. This approach removes a whole class of bugs caused by different versions of the logic drifting apart over time. Keeping the logic agnostic of where it runs can’t rely on developer discipline. The architecture must make the right way the easiest way to write a workflow. Whenever someone writes non-agnostic code, the build must fail. The decoupling isn’t free. The orchestration loses direct access to each runtime’s native features. Every new capability has to be wired through the agnostic layer. This…