Optimizing cost and latency with Amazon Bedrock prompt caching

Prompt caching in Amazon Bedrock can reduce your input token costs by up to 90 percent when you repeatedly send the same context to foundation models, based on Amazon Bedrock prompt caching pricing. Without caching, a 10,000-token contract sent alongside 50 user questions means 500,000 input tokens billed at full price for content the model has already processed.

You can mitigate this issue by shortening prompts, reducing context windows, or implementing application-level caching. Each option involves a trade-off:

Shortened prompts reduce token count but might also reduce context quality.

Smaller context windows lower cost at the expense of the model’s ability to reason over complete information.

Response caching handles identical queries well, yet provides no benefit when the same context is paired with different questions.

Prompt caching in Amazon Bedrock helps reduce this challenge at the infrastructure level. When you cache parts of your conversation context (system prompts, documents, tool definitions), Amazon Bedrock reads the cached tokens on subsequent requests instead of reprocessing them. This can reduce time-to-first-token (TTFT) and lower costs for cached input tokens by up to 90 percent on cache hits, without changing your model or prompt quality.

Message content caching: Cache long documents for multi-question analysis.

System prompt caching: Cache persona definitions and instructions across conversations.

Tool definition caching: Cache tool schemas for agentic workflows.

Mixed TTL caching: Assign different cache lifetimes to different content tiers.

Tenant isolation: Implement per-tenant cache separation in multi-tenant applications.

LangChain integration: Use prompt caching with the LangChain framework.

Prompt caching stores a snapshot of partially processed input so that subsequent requests with the same prefix skip redundant computation. This section covers the request flow, supported models, and pricing.

When you include a cachePoint marker in your request, Amazon Bedrock evaluates whether the content preceding that marker matches an existing cache entry. If it does (a cache hit), the model can skip reprocessing those tokens and begin generation from the cached state. If no match exists (a cache miss), the model processes the full content and writes the result to cache for potential future requests.

Cache scope: Cache entries are scoped to individual AWS accounts and AWS Regions.

Token thresholds: Each cache checkpoint must meet a minimum token threshold to activate. For example, Anthropic Claude Sonnet 4.5 and Sonnet 4.6 require at least 1,024 tokens per checkpoint, while Opus models require at least 4,096.

Time-to-live (TTL): Cache entries expire based on the TTL specified in the request. The default is 5 minutes, with select models supporting up to 1 hour.

Model-agnostic syntax: The Converse API cachePoint syntax is identical across supported model families, including Anthropic Claude and Amazon Nova.

For the latest model support information, see the Amazon Bedrock Prompt Caching documentation.

For workloads with repeated context, the savings reach approximately 75 percent on input token costs. For example, if you send a 10,000-token document with 10 different questions, the first request incurs a cache write cost. The remaining nine requests each read from cache at 90 percent reduced cost, resulting in a net savings of approximately 75 percent on input token costs for that document context. This assumes all subsequent requests occur within the TTL window. Requests after expiration trigger a new cache write, reducing the net savings. See Amazon Bedrock pricing for detailed pricing information.

An AWS account with Amazon Bedrock access in a supported AWS Region (such as us-west-2).

Model access enabled for the target model. The examples in this post use Anthropic Claude Sonnet 4.5 (global.anthropic.claude-sonnet-4-5-20250929-v1:0). See Manage model access for instructions. For the latest model and Region availability, see Supported models by AWS Region in Amazon Bedrock. This is a cross-Region inference profile. Requests automatically route across Regions, which can occasionally increase cache write frequency.

Python 3.10 or later with the following dependencies installed: pip install boto3>=1.43.0 langchain-aws>=0.2.12 matplotlib pandas

Note: Boto3 1.43.0 or later is required for the ttl parameter in cachePoint used in Scenario 4 (Mixed TTL).

Note: Boto3 1.43.0 or later is required for the ttl parameter in cachePoint used in Scenario 4 (Mixed TTL).

AWS credentials configured through the default profile or environment variables. See Configure the AWS Command Line Interface (AWS CLI) for setup instructions.

A common use case for prompt caching is caching long documents or reference content that you query repeatedly. For example, in a Retrieval Augmented Generation (RAG) application, you ask multiple questions about the same document, or a coding assistant references a large codebase.

In this scenario, you place a cachePoint marker between the static document and the dynamic question. Amazon Bedrock caches the document on the first call and might reuse it on subsequent calls.

Place a cachePoint content block after the static content and before the dynamic question. Amazon Bedrock caches everything before the checkpoint and reuses it on subsequent requests:

The following code puts this pattern into practice with a complete working example.

First, set up the Amazon Bedrock runtime client and define a sample document. In a production application, this document can be a PDF, a knowledge base article, or other content exceeding the 1,024-token threshold:

Next, define the caching function. The key elements are the cachePoint block placed between the static document and the dynamic question:

Run two requests to observe the caching behavior. The initial call populates the cache, and a subsequent call with a different question reuses it:

In our testing with Anthropic Claude Sonnet 4.5 and a document exceeding 1,024 tokens, the initial response shows a cache write:

Notice that cacheReadInputTokens now reflects the 1,898 tokens read from cache. The entire document prefix was reused without reprocessing. Only 28 tokens (the question itself) were processed as standard input. These cached tokens are billed at the reduced cache-read rate (90 percent lower than standard input).

Claude models on Amazon Bedrock support simplified cache management. You can place a single cachePoint, and Amazon Bedrock automatically checks for cache hits on prefixes up to approximately 20 content blocks before that marker. You do not need to manually place multiple cache checkpoints to get cache hits on earlier portions of your conversation.

This approach supports partial cache hits. If only the first two sections match a previous request, the model reuses the cache for those sections and processes the remaining content.

The same caching syntax works with converse_stream. The key difference is that cache metrics arrive in the metadata event at the end of the stream rather than in the immediate response:

Prompt caching can reduce TTFT, with the benefit growing as the cached prefix size increases. For smaller documents (approximately 2,000–5,000 tokens), the improvement may not be statistically significant across a small number of iterations. The exact improvement varies based on document size, model, and current load. For large cached prefixes (over 10,000 tokens), the reduction in TTFT becomes more pronounced.

Many applications use detailed system prompts that define the model’s persona, guidelines, and domain expertise. These system prompts can span thousands of tokens and remain constant across user interactions. With system prompt caching, you pay the full processing cost once and reuse the cached system prompt for every subsequent message.

The cache point goes inside the system parameter, separate from user messages.

The Converse API system parameter accepts an array of content blocks. Place a cachePoint after the system text:

The next example shows this in a full request with a detailed persona prompt.

The following example defines a comprehensive system prompt: an Expert Space Science Advisor persona with detailed response guidelines. The prompt exceeds the 2,048-token threshold required for caching:

The user message changes between requests, but the system prompt remains identical. Amazon Bedrock is designed to cache the system prompt prefix and reuse it, regardless of what the user asks.

Persona-based assistants with detailed role descriptions and behavioral guidelines.

Agentic workflows with extensive instructions that stay constant across turns.

Customer service bots with complex company policies and response protocols.

Domain-specific assistants with embedded knowledge bases in the system prompt.

Agentic applications often define dozens of tools with comprehensive JSON schemas. These tool definitions can collectively contain thousands of tokens and rarely change between requests. Tool definition caching prevents reprocessing these schemas on every turn.

The example below demonstrates this with a set of tool schemas that collectively exceed the token threshold.

Here we define a set of space-themed tools with comprehensive schemas. In a production application, these might be API integrations, database queries, or external service calls:

Tool definition caching is useful for agentic workflows where the same set of tools is invoked across many conversation turns. By caching the tool schemas once, you avoid reprocessing thousands of tokens of schema definitions on every turn.

Cached content doesn’t have a single lifecycle. Core reference material (domain knowledge, product catalogs, compliance rules) rarely changes and benefits from longer cache durations. Session-specific context (recent conversation turns, user preferences) changes more frequently and benefits from shorter expiration. With mixed TTL caching, you can assign different expiration times to different content tiers within a single request.

Each cachePoint can include a ttl field. There is one ordering constraint: longer TTL checkpoints must appear before shorter ones in the request:

The Converse API response includes a cacheDetails field that shows the per-TTL token breakdown. This shows whether both TTL tiers are working correctly. When both sections independently exceed the model’s minimum token threshold, you will see one entry per TTL:

If only one section exceeds the threshold, the response will show a single TTL entry. Each content tier must independently meet the model’s minimum token requirement to get separate cache entries for each TTL.

Cache checkpoints must be ordered from longest to shortest TTL within a single request. The API returns an error if a shorter TTL appears before a longer one:

✅ Valid: 1h then 5m.

❌ Invalid: 5m then 1h.

This constraint applies across cache checkpoint locations (messages, system prompt, and tool definitions).

With mixed TTL in place, the next scenario addresses a common multi-tenant challenge: preventing one tenant’s cached content from being read by another.

In multi-tenant applications, you must prevent one tenant’s cached content from being read by another tenant. Prompt caching in Amazon Bedrock scopes entries by account and Region, but within the same account and Region, cache entries might be shared across requests. The SHA-256 hash prefix pattern provides tenant isolation with only approximately 16 tokens of overhead, without requiring separate AWS accounts.

Prepend a SHA-256 hash of the tenant_id to the cached content. Because the hash changes the content prefix, Amazon Bedrock creates a separate cache entry for each tenant:

No server-side configuration: You achieve isolation purely through content prefixing.

Independent cache entries per tenant: Each tenant’s cached content stays separate.

Minimal overhead: The SHA-256 hash adds only 64 characters (approximately 16 tokens) to the prompt.

For teams using the LangChain framework, prompt caching integrates with the ChatBedrockConverse class. LangChain provides a create_cache_point() method that generates the correct cachePoint content block without requiring you to manage the raw API format.

Use ChatBedrockConverse.create_cache_point() within message content arrays or ChatPromptTemplate definitions:

The scenarios in this post use the Converse API, which provides a model-agnostic cachePoint syntax. If your application uses the InvokeModel API, be aware that the caching syntax differs by model family:

Use the Converse API for new applications because the cachePoint syntax works identically across the supported model families. This means you can switch between Anthropic Claude and Amazon Nova without changing your caching code.

Profile your prompts: Identify which components are static (system prompts, tool schemas, reference documents) and which are dynamic (user questions, session context). Cache the static components.

Meet the token threshold: Each cache checkpoint must exceed the model’s minimum token requirement. Amazon Bedrock processes content below the threshold normally without caching.

Choose appropriate TTLs: Use 1-hour TTLs for rarely-changing content (domain knowledge, tool definitions) and 5-minute TTLs for session-specific context. Remember that longer TTLs must appear before shorter ones.

Monitor cache metrics: Track cacheWriteInputTokens and cacheReadInputTokens in your application logs. A low cache-hit ratio might indicate that your content is changing too frequently or that your TTL is too short.

Apply responsible AI controls: For production deployments, use Amazon Bedrock Guardrails to add content filtering and grounding validation alongside your caching patterns.

Implement tenant isolation for multi-tenant systems: Use the SHA-256 hash prefix pattern to prevent cross-tenant cache sharing when serving multiple customers from the same AWS account.

Use simplified cache management: For Anthropic Claude models, a single cachePoint can cover multiple preceding content blocks. You don’t always need to place checkpoints after every section.

Combine caching locations: You can cache system prompts, message content, and tool definitions simultaneously in a single request. This stacks the savings across the three locations.

The examples in this post use on-demand Amazon Bedrock inference and don’t create persistent AWS resources. No cleanup is required beyond stopping any running Jupyter notebook kernels.

This post walked through six practical prompt caching scenarios using the Amazon Bedrock Converse API, progressing from basic document caching to advanced patterns like mixed TTL and tenant isolation. Each scenario addresses a specific production challenge:

Cache content to reduce costs when you ask multiple questions about the same document.

Reuse the prompt and avoid reprocessing detailed persona definitions on every conversation turn.

Optimize agentic workflows by efficiently caching tool schemas once.

Tune mixed TTLs for fine-grained control over cache lifetimes for content with different update frequencies.

Isolate tenant caches for safe multi-tenant deployments with per-tenant cache separation.

Integrate with LangChain to bring prompt caching to your framework with minimal code changes.

The cachePoint syntax is model-agnostic across the Converse API. The same code works with Anthropic Claude, Amazon Nova, and other supported models that support prompt caching on Amazon Bedrock. You can adopt prompt caching incrementally, starting with the highest-impact scenario for your workload and expanding to additional patterns as needed.

For next steps, profile your existing Amazon Bedrock applications to identify opportunities for prompt caching. Start with the scenario that matches your workload (document analysis, persona-based assistants, or agentic tool use) and measure the TTFT and cost improvements.

Each scenario includes working code that you can run immediately against Amazon Bedrock. The complete set of notebooks and scripts referenced throughout this post is available in the amazon-bedrock-samples GitHub repository. For more details on prompt caching configuration and supported models, see the Amazon Bedrock Prompt Caching documentation.

To get started, visit Amazon Bedrock or open the Amazon Bedrock console to enable model access and begin using prompt caching in your applications.