Amazon Bedrock Prompt Caching: Reduce Input Costs by 90%

Amazon Bedrock prompt caching allows developers to reduce input token costs by up to 90% and significantly lower Time-to-First-Token (TTFT) latency by reusing frequently accessed context. By inserting a checkpoint marker in the Converse Application Programming Interface (API), Amazon Bedrock stores a snapshot of processed tokens—such as system prompts, tool definitions, or large documents—to avoid redundant computation on subsequent requests.

How Amazon Bedrock Prompt Caching Works

Prompt caching functions at the infrastructure level by storing the state of a partially processed prompt prefix. When a request includes a cachePoint marker, the Amazon Bedrock runtime checks if the preceding content matches an existing entry in the cache.

If the content matches, a "cache hit" occurs. The model skips reprocessing those tokens and begins generation immediately from the cached state. If there is no match, a "cache miss" occurs; the model processes the full content and writes the result to the cache for future use. This mechanism is particularly effective for AI coding agents that reference the same codebase across multiple turns.

Key Concepts and Constraints

• Cache Scope: Entries are strictly isolated to individual Amazon Web Services (AWS) accounts and AWS Regions.
• Token Thresholds: Caching only activates if the content preceding the cachePoint meets a minimum token count. For Anthropic Claude 3.5 Sonnet, this is 1,024 tokens. For Claude 3 Opus, the threshold is 4,096 tokens.
• Time-to-Live (TTL): The default cache duration is 5 minutes. Certain models allow developers to extend this to 1 hour for a higher write fee.
• Model Support: The feature uses model-agnostic syntax in the Converse API, supporting families like Anthropic Claude and Amazon Nova.

Pricing and Cost Efficiency

Prompt caching introduces a tiered billing structure for input tokens. While the initial "write" to the cache is more expensive than a standard input token, every subsequent "read" from the cache is billed at a 90% discount.

For a workload where a 10,000-token document is queried 10 times, the first request incurs a 1.25x write cost. The following nine requests use the 0.1x read cost. According to AWS, this results in a net savings of approximately 75% on total input token costs for that specific context, provided all requests occur within the TTL window. Developers should monitor these shifts closely, especially as AI model API pricing continues to evolve.

Implementing Cache Points in the Converse API

To enable caching, developers must modify the messages or system blocks in their API calls. The cachePoint marker should be placed immediately after the static content and before the dynamic user input.

Message Content Caching

This pattern is ideal for Retrieval Augmented Generation (RAG) applications where users ask multiple questions about the same large document.

content = [
    {"text": "<large_static_document_content>"},
    {"cachePoint": {"type": "default"}}, # Snapshot everything above
    {"text": "What are the key findings in this report?"} # Dynamic question
]

System Prompt Caching

For complex personas or rigid output guidelines, the system prompt can be cached. This is useful when migrating high-performance workloads, such as those using GPT-6 Astra on Amazon Bedrock, where system instructions are extensive.

system = [
    {"text": "<detailed_persona_and_guidelines>"},
    {"cachePoint": {"type": "default"}}
]

Tool Definition Caching

In agentic workflows, tool schemas often consume a significant portion of the prompt. Caching these definitions ensures that the model doesn't re-parse the Model Context Protocol (MCP) schemas or custom tool JSON on every turn.

Performance Benchmarks: TTFT Reduction

The latency benefit of prompt caching is measured primarily through Time-to-First-Token (TTFT). By eliminating the need to re-compute the hidden states of the prompt prefix, the model can begin generating the response faster.

AWS testing indicates that while latency improvements for small prefixes (under 5,000 tokens) may be negligible, the reduction in TTFT becomes "pronounced" for prefixes exceeding 10,000 tokens. This makes prompt caching a critical optimization for long-context windows where the model must reason over entire codebases or legal contracts.

Actionable Integration Strategy

To maximize the ROI of Amazon Bedrock prompt caching, developers should categorize their workloads by context stability:

1. High-Frequency RAG: If users typically ask 3+ questions per document, implement message content caching. The 90% discount on subsequent reads will quickly offset the 25% write premium.
2. Complex Agents: For agents with more than 1,024 tokens of system instructions or tool definitions, move the cachePoint to the end of the system block. This ensures every interaction in a multi-turn conversation benefits from reduced latency.
3. Multi-Tenant Applications: Use specific prefixing to ensure tenant isolation. Since cache hits require an exact prefix match, ensure that tenant-specific data is placed before a cachePoint only if that cache is intended to be shared among users of that specific tenant.
4. Long-Running Sessions: For applications where user interactions are spaced more than 5 minutes apart, utilize the 1-hour TTL option for supported models. Although the write cost doubles, it prevents frequent "cache churn" where entries expire before they can be reused.