RetrieveAndGenerate vs Retrieve-Then-Prompt

Managed RAG orchestration vs building your own retrieval-augmentation loop.

Every Knowledge Base query eventually needs to turn retrieved passages into a generated answer. Bedrock gives you two ways to get there: let it manage the whole thing with RetrieveAndGenerate, or call Retrieve yourself and build the prompt with Converse. This page runs the same question through both so the trade-off is concrete, then gives you a rule for choosing.

Recipe

Quick-reference recipe card - copy-paste ready.

# --- Python (boto3) ---
import boto3
 
agent_rt = boto3.client("bedrock-agent-runtime", region_name="us-east-1")
 
# RetrieveAndGenerate: one call, managed prompt, built-in citations.
resp = agent_rt.retrieve_and_generate(
    input={"text": "What is our refund policy?"},
    retrieveAndGenerateConfiguration={
        "type": "KNOWLEDGE_BASE",
        "knowledgeBaseConfiguration": {
            "knowledgeBaseId": kb_id,
            "modelArn": "arn:aws:bedrock:us-east-1::foundation-model/amazon.nova-lite-v1:0",
        },
    },
)
print(resp["output"]["text"])

// --- TypeScript (AWS SDK v3) ---
import { BedrockAgentRuntimeClient, RetrieveAndGenerateCommand } from "@aws-sdk/client-bedrock-agent-runtime";
 
const agentRt = new BedrockAgentRuntimeClient({ region: "us-east-1" });
 
// RetrieveAndGenerate: one call, managed prompt, built-in citations.
const resp = await agentRt.send(new RetrieveAndGenerateCommand({
  input: { text: "What is our refund policy?" },
  retrieveAndGenerateConfiguration: {
    type: "KNOWLEDGE_BASE",
    knowledgeBaseConfiguration: {
      knowledgeBaseId: kbId,
      modelArn: "arn:aws:bedrock:us-east-1::foundation-model/amazon.nova-lite-v1:0",
    },
  },
}));
console.log(resp.output?.text);

When to reach for each:

• RetrieveAndGenerate - the fastest path to a working RAG answer, with citations already parsed out for you.
• Retrieve-then-prompt - you need a custom prompt structure, want to mix retrieved passages with other context (tool results, conversation history), or want to send the final prompt to a different generation model or a non-Bedrock provider.
• Default to RetrieveAndGenerate; drop to retrieve-then-prompt only when you have a concrete reason.

Working Example

Here is the same question answered both ways: first the managed one-call path, then the two-step path where you build the Converse prompt yourself from raw retrieved passages.

# --- Python (boto3) ---
import boto3
 
agent_rt = boto3.client("bedrock-agent-runtime", region_name="us-east-1")
brt = boto3.client("bedrock-runtime", region_name="us-east-1")
question = "What is our refund policy?"
 
# 1. RetrieveAndGenerate - managed prompt and generation in one call.
managed = agent_rt.retrieve_and_generate(
    input={"text": question},
    retrieveAndGenerateConfiguration={
        "type": "KNOWLEDGE_BASE",
        "knowledgeBaseConfiguration": {
            "knowledgeBaseId": kb_id,
            "modelArn": "arn:aws:bedrock:us-east-1::foundation-model/amazon.nova-lite-v1:0",
        },
    },
)
print("Managed:", managed["output"]["text"])
 
# 2. Retrieve-then-prompt - you own the prompt.
passages = agent_rt.retrieve(
    knowledgeBaseId=kb_id,
    retrievalQuery={"text": question},
    retrievalConfiguration={"vectorSearchConfiguration": {"numberOfResults": 3}},
)["retrievalResults"]
 
context = "\n\n".join(p["content"]["text"] for p in passages)
prompt = (
    f"Answer using only the context below. If the answer isn't there, say so.\n\n"
    f"Context:\n{context}\n\nQuestion: {question}"
)
custom = brt.converse(
    modelId="amazon.nova-lite-v1:0",
    messages=[{"role": "user", "content": [{"text": prompt}]}],
    inferenceConfig={"maxTokens": 300},
)
print("Custom:", custom["output"]["message"]["content"][0]["text"])

// --- TypeScript (AWS SDK v3) ---
import { BedrockAgentRuntimeClient, RetrieveCommand, RetrieveAndGenerateCommand } from "@aws-sdk/client-bedrock-agent-runtime";
import { BedrockRuntimeClient, ConverseCommand } from "@aws-sdk/client-bedrock-runtime";
 
const agentRt = new BedrockAgentRuntimeClient({ region: "us-east-1" });
const brt = new BedrockRuntimeClient({ region: "us-east-1" });
const question = "What is our refund policy?";
 
// 1. RetrieveAndGenerate - managed prompt and generation in one call.
const managed = await agentRt.send(new RetrieveAndGenerateCommand({
  input: { text: question },
  retrieveAndGenerateConfiguration: {
    type: "KNOWLEDGE_BASE",
    knowledgeBaseConfiguration: {
      knowledgeBaseId: kbId,
      modelArn: "arn:aws:bedrock:us-east-1::foundation-model/amazon.nova-lite-v1:0",
    },
  },
}));
console.log("Managed:", managed.output?.text);
 
// 2. Retrieve-then-prompt - you own the prompt.
const retrieved = await agentRt.send(new RetrieveCommand({
  knowledgeBaseId: kbId,
  retrievalQuery: { text: question },
  retrievalConfiguration: { vectorSearchConfiguration: { numberOfResults: 3 } },
}));
const passages = retrieved.retrievalResults ?? [];
 
const context = passages.map((p) => p.content?.text).join("\n\n");
const prompt = `Answer using only the context below. If the answer isn't there, say so.\n\nContext:\n${context}\n\nQuestion: ${question}`;
 
const custom = await brt.send(new ConverseCommand({
  modelId: "amazon.nova-lite-v1:0",
  messages: [{ role: "user", content: [{ text: prompt }] }],
  inferenceConfig: { maxTokens: 300 },
}));
console.log("Custom:", custom.output?.message?.content?.[0]?.text);

What this demonstrates:

• RetrieveAndGenerate never exposes the intermediate prompt - you get output.text and citations directly.
• Retrieve gives you raw passages (content.text, score, location) that you assemble into your own prompt string.
• The custom path uses bedrock-runtime (Converse) as a second, separate client from bedrock-agent-runtime - retrieval and generation are decoupled.
• Both paths can use the same generation model; the difference is who constructs the prompt and who owns its exact wording.

Deep Dive

What RetrieveAndGenerate Manages For You

RetrieveAndGenerate runs retrieval, builds a prompt that instructs the model to answer from the retrieved passages, calls the generation model, and parses citations that map parts of the answer back to specific retrieved passages. All of that - the prompt template, the citation extraction - is Bedrock's implementation, not yours to inspect or edit line by line. You configure what it retrieves from and which model generates, not the literal prompt text.

What You Control With Retrieve-Then-Prompt

Calling Retrieve directly gives you the passages and nothing else - no prompt, no generation. You then own every subsequent decision: how to format the context, what system instructions to add, whether to include conversation history, whether to mix in results from a tool call or a second Knowledge Base, and which model (any Bedrock model, via Converse) actually generates the answer. This is strictly more work, and strictly more control.

Citations: Managed vs Manual

RetrieveAndGenerate's citations array is a meaningful advantage - it maps generated text back to source passages automatically, which is exactly the kind of bookkeeping you'd otherwise have to build yourself (asking the model to cite sources reliably in free text is unreliable). If you go the retrieve-then-prompt route and need citations, budget time to build that yourself - for example, by asking the model to reference passage numbers you assign, then mapping them back in your own code.

Mixing Non-Retrieved Context

RetrieveAndGenerate is scoped to what it retrieves from the configured Knowledge Base (or Knowledge Bases, if you configure more than one). It has no mechanism for injecting arbitrary extra context - a tool result, a user's account data fetched separately, prior conversation turns from a different system. Retrieve-then-prompt has none of that limitation, because you are building the final Converse messages array yourself and can put anything in it.

Streaming

Both paths support streaming. RetrieveAndGenerateStream (or the stream variant of the same operation) streams the generated answer as it is produced; the retrieve-then-prompt path streams via ConverseStream after your own synchronous Retrieve call. Neither streams the retrieval step itself - retrieval is a fast, single round trip either way, and streaming benefits apply to the generation half.

Gotchas

• Expecting RetrieveAndGenerate to accept extra context - it only uses what it retrieves from the configured Knowledge Base. Fix: use retrieve-then-prompt when you need to mix in tool results, other data, or conversation history.
• Rebuilding citation logic when RetrieveAndGenerate already provides it - reinventing source attribution is unnecessary work. Fix: default to RetrieveAndGenerate specifically when citations matter and you have no other reason to go manual.
• Assuming retrieve-then-prompt is always more expensive - it is not inherently; cost is driven by tokens and calls either way, and a well-scoped custom prompt can be leaner than a generic managed one. Fix: compare actual token usage on your workload rather than assuming.
• Forgetting retrieve-then-prompt is two separate clients - bedrock-agent-runtime for Retrieve, bedrock-runtime for Converse. Fix: import and construct both clients; they are not interchangeable.
• Not instructing the model to stay within the retrieved context - a hand-built prompt without that instruction can let the model answer from its own training knowledge instead of your documents. Fix: explicitly instruct "answer using only the context below" (or similar) in your prompt.

Alternatives

FAQs

Related

• Bedrock Knowledge Bases Basics - the first Retrieve and RetrieveAndGenerate calls.
• The Agentic Retriever for Multi-Step Queries - orchestration on top of either pattern for complex questions.
• Managed RAG on Bedrock: What the Knowledge Base Owns - where both patterns sit in the pipeline.
• Bedrock Knowledge Bases Best Practices - retrieval-quality and citation-handling recommendations.

Stack versions: This page was written for boto3 1.43.x (Python 3.10+) and the AWS SDK for JavaScript v3 (Node.js 18+). Bedrock Knowledge Bases' Agentic Retriever is a fast-evolving 2026 capability - verify exact parameters against current docs.