Agentic retrieval for Amazon Bedrock Managed Knowledge Base

Your users ask multi-part, comparative, and exploratory questions that span PDFs, slides, tickets, transcripts, and web content. Classic single-shot retrieval breaks down on these questions. Answers miss context, support tickets escalate, and analysts waste hours re-running searches. Agentic retrieval for Amazon Bedrock Managed Knowledge Bases is designed for these questions.

Consider two questions an analyst might ask: "Compare our 2020 and 2023 strategy. What changed?" and "What are the three biggest risks across product lines?" Neither is a single-shot lookup. A multi-intent question has no single point in embedding space that represents it well, so top-k chunks come back as an average of competing sub-intents.

Agentic retrieval plans and iterates over retrieval and can generate a response in the same call. This post focuses on why classic retrieval falls short on multi-part questions, how the AgenticRetrieveStream API works (including request construction and trace parsing), and when to choose it over the standard Retrieve API.

A worked example makes the gap concrete. Suppose you have ingested 25 years of Amazon shareholder letters into a Managed Knowledge Base. Your first question is direct:

"What is the most important message in the documents?"

The standard Retrieve API returns five chunks by hybrid score. In our test corpus, the top result is high-scoring but low-value: it covers the Amazon logo color scheme. The following chunks cover hiring, builders, and differentiation. The retriever did its job and ranked by similarity to the query embedding. But "most important message" is vague, and the scorer has no signal for what matters.

"Compare how Amazon talked about hiring, long-term investment, and customer obsession in 2020 vs 2023. Where did emphasis shift?"

Single-shot retrieval must now serve three intents across two time periods with one query vector. You get one of two outcomes: a scatter of loosely related chunks, or a tight cluster dominated by the strongest signal. Neither is what a human analyst would do.

A human would decompose the question. They would search hiring in 2020, then hiring in 2023, then investment philosophy in each year. They would read the results, notice gaps, refine the search, and run it again. With enough evidence in hand, they would synthesize.

Agentic retrieval automates this workflow as an API.

Multi-part questions: A single embedding may not represent a multi-intent query well, so top-k results tend to blend intents together.

Comparative reasoning: "Compare A and B across dimension X" requires targeted retrieval per item, not one blended search.

Cross-source questions: When evidence spans multiple knowledge bases, static routing rules are brittle and might not adapt well per query.

Sufficiency: Classic retrieval doesn't judge "enough." It returns k chunks regardless.

Exploration: Some questions need follow-up searches based on what the first results reveal. Single-shot retrieval can't iterate.

Teams have addressed this gap with custom agent frameworks built on the Retrieve API. They prompt a model to plan and call Retrieve API in a loop, manage sub-queries, deduplicate, and decide when to stop. It works, but it becomes custom plumbing in every application, each instance carrying its own latency, cost, and reliability risks.

Agentic retrieval is a new retrieval mode in Amazon Bedrock Managed Knowledge Bases, available through the AgenticRetrieveStream API. Instead of one similarity search, it runs a planning loop driven by a foundation model: the model decomposes your question, retrieves against each part, judges whether it has enough evidence, and iterates as needed. It generates a grounded response by default in the same call. Set generateResponse=False to return retrieval results without generating a response.

You see every step of that loop as an ordered stream of trace events. Each event carries a step and a status, so you can see what the planner did and why:

SpeculativeRetrieval: An initial retrieval that runs before the first planning step to reduce end-to-end latency. With a single KB, it uses the raw user query. With multiple KBs, it is a probe search used to route sub-queries. It doesn't count toward maxAgentIteration.

Planning: The foundation model (FM) analyzes the query, reviews prior results, and emits sub-queries aligned to retrievable intents. Between iterations, this is also where the model judges sufficiency and decides whether to stop or search again.

Retrieval or FullDocumentExpansion: One event per sub-query execution, each with its own status. In agentic retrieval, this action/event can fetch a full document if the agent determines that the passage context is insufficient to answer the query. Often, queries that require summarization, listing elements, or information from various parts of a document benefit from this action. The loop may run several iterations if more evidence is needed, bounded by maxAgentIteration.

Result: The final event. It contains the deduplicated source chunks collected across the iterations. For multi-KB requests, each chunk identifies its sourceRetriever. Because response generation is enabled by default, the result event also contains the grounded answer and citations unless you set generateResponse=False.

A note on deduplication and the trace. Deduplication applies only to the final result event. When multiple sub-queries retrieve the same chunk, that chunk appears only once in the final results. Trace events remain available for reviewing the individual planning and retrieval steps.

Two parameters do most of the tuning. The first is maxAgentIteration, the ceiling on planning-plus-retrieval iterations. Use 3 for a single KB and 4–5 for multi-KB or comparative queries, though the planner may exit early after its evaluation step finds the evidence sufficient. See Quotas and limits for defaults and maximums.

The second is the foundation model configuration. The foundationModelType field determines whether agentic retrieval uses the service-managed model or a custom model. When you set it to CUSTOM, provide the model through foundationModelConfiguration.

Note on pricing: Agentic retrieval is priced per call, and the model you choose for the orchestration sets the rate. With the managed model, you pay $4 per 1,000 agentic retrieval calls plus $1 per 1,000 underlying Retrieve API calls. If you choose a model available in Amazon Bedrock, you pay standard pass-through Bedrock pricing for that model plus the same $1 per 1,000 underlying Retrieve calls. See more details on the pricing page.

Now that you have seen how the planning loop works, we examine the AgenticRetrieveStream API in practice.

The most direct case: one KB, one query, let the planner decompose. You send the request, then consume the stream, which delivers trace events, generated-response chunks when response generation is enabled, and a final result event.

That is the whole loop: one request, no orchestration code, and a stream you read top to bottom. The trace events tell you what the planner did. The result event gives you the deduplicated chunks. In production, log these trace events to Amazon CloudWatch for cost attribution, latency budgeting, and debugging. Each iteration is one planner invocation plus its sub-query retrievals, so the number of iterations is what drives both cost and wall-clock latency.

Multi-KB routing is what agentic retrieval does that the other APIs cannot. Register up to five Managed KB retrievers in one request and attach a natural-language description to each. The planner reads those descriptions to route each sub-query to the most relevant KB.

You consume the stream the same way. The routing decision shows up in the result: ea