Skip to content
aiqbenchAI & Quantum Intelligence
NewsInsightsBenchmarksSkillsAsk AILearnAbout
...Loading your session
NewsInsightsBenchmarksSkillsAsk AILearnAbout
Back to list
aiqbench takeChips & Infra

aiqbench analysis

This incident report details a critical regression in LiteLLM's handling of Anthropic prompt caching when routing Claude Code requests through Amazon Bedrock, causing significant cost increases for users on affected versions. The root cause was an undocumented structural change to system message placement that invalidated cache breakpoints, highlighting the risks of proxy software failing to preserve provider-specific semantic nuances like caching semantics. This event underscores the importance of rigorous regression testing and transparent documentation when managing AI infrastructure gateways where silent cost regressions can occur without breaking functional correctness.
1
Source coveringLiteLLM Blog

Incident Report: Prompt Cache Invalidation for Claude Code on Bedrock Invoke

LiteLLM Blog·July 13, 2026·9 min read

From LiteLLM Blog

Excerpt

Date: July 4 to July 10, 2026 Affected versions: v1.91.0 and v1.91.1 Severity: Medium (silent cost regression; no correctness impact) Status: Resolved in v1.91.2

Note: If you run Claude Code against Amazon Bedrock through LiteLLM on either v1.91.0 or v1.91.1, upgrade to v1.91.2 or higher.

Between July 4 and July 10, proxies running v1.91.0 or v1.91.1 silently broke Anthropic prompt caching for Claude Code sessions routed through Amazon Bedrock's Invoke API. For the customers who reported it, warm-session cache hit rates dropped from roughly 90% to 25-45% and team daily spend rose 2-3x for the same usage. Requests kept returning 200s with correct completions; the only symptoms were the cache miss rate and the bill.

The cause: PR #31364 moved every role: "system" entry in messages into the top-level system field on the Invoke path, which invalidates every cache breakpoint past the tool definitions and system prompt. The fix shipped July 10 in v1.91.2 (#32578, #32831, #32882), with regression tests that fail on pre-fix code.

We own this outcome entirely.

Excerpt shown under the publisher's RSS terms · 170 words · full text and figures are on the publisher's site

Read at LiteLLM Blog ↗Ask about this article

Related coverage

Quantum Comms

WISeQey Advances PKI Modernization as Recent Cyberattacks Highlight the Need to Limit Identity and Credential Risks

WISeKey · Nov 5, 2026

Chips & Infra

RIKEN Warns Fugaku and RIKYU Users of Possible Personal Data Exposure

HPCwire · Nov 18, 2026

Critical SonicWall Remote Code Execution Vulnerabilities Actively Exploited in Attacks

Cybersecurity News · Sep 2, 2026

New 2CLoader Malware Evades Security Tools to Deploy Vidar and Remus Stealers

Cybersecurity News · Oct 1, 2026

Article info

Source

LiteLLM Blog

Published

July 13, 2026

Read time

9 min

Category

Chips & Infra
aiqbench
AI & Quantum Intelligence© aiqbench. All rights reserved.

Feeds:RSS·JSON Feed

LanguageEnglish·Românăbeta
NewsAI & TechQuantumInsights
ExploreBenchmarksLearn
CompanyAboutTrustLegal noticeTerms of ServicePrivacy PolicyCopyright noticesData erasureReport content

The aiqbench take, the insights, the daily digest and Ask answers are written by AI models from the sources shown, without editorial review before publication. They may contain errors — check the source. How the model is used →

  • News
  • Insights
  • Ask AI
  • Benchmarks
  • Account