Skip to content
aiqbench takeAI Products

aiqbench analysis

This article details a practical cost-optimization technique for LLM agents by combining prompt caching with sticky routing to avoid paying full rates on repeated system instructions. By pinning sessions to specific provider endpoints, developers can reduce token costs significantly while managing the trade-off between cache write overhead and read savings.
Source coveringOpenRouter
What this isA model's reading of the article and related coverage, written for aiqbench. Not the publisher's words.

The Cheapest Token Is a Cached One: Prompt Caching + Sticky Routing

OpenRouter11 min read

Your agent sends the same system prompt, tool definitions, and schemas on every turn. Cache reads cost 0.1x to 0.5x of fresh input, but only if the next request lands on the provider holding the warm cache. Here's how caching and sticky routing work together, and how to confirm they're working.

Your agent sends the same system prompt, tool definitions, schemas, and policy instructions on every turn. In a 6-turn session, you can get billed for that same opening block 6 times, even though the only thing that changed is the user’s latest message or the agent’s latest tool result. Prompt caching fixes that. The provider reads the repeated part of your prompt from cache instead of charging you full price for it every time. Sticky routing keeps that working across turns by sending a session back to the same provider that holds the warm cache. This post covers the money side: what cached tokens cost, why cache reads and writes are priced differently, how session_id keeps an agent’s session warm from turn one, and how to check that caching is actually working. A cache read costs 0.1x to 0.5x of a fresh input token, depending on the provider. On Claude Sonnet 4.6, a cache read runs at $0.30/M against $3.00/M input, exactly 0.1x. The first request pays a cache write. Anthropic writes cost 1.25x (5-minute TTL) or 2.0x (1-hour TTL) of input, so a single unreused write costs more than not caching at all. A warm cache only helps if…

Excerpt shown under the publisher's RSS terms · 200 words · full text and figures are on the publisher's site

Related coverage