Managed RAG on Bedrock: What the Knowledge Base Owns

Ingestion, chunking, embedding, and retrieval as one managed pipeline.

Retrieval-Augmented Generation (RAG) means grounding a model's answer in your own documents instead of relying on what it memorized during training. Building that yourself means writing a document loader, a chunker, an embeddings caller, a vector database client, and a retrieval-then-prompt loop. A Bedrock Knowledge Base replaces all of that with one managed resource. This page builds the mental model for what it owns, what you still configure, and which client you reach for at each stage.

Summary

• A Knowledge Base is a managed RAG pipeline: point it at source documents, choose a vector store, and Bedrock handles chunking, embedding, indexing, and querying.
• Insight: the pipeline is split across two distinct SDK client families - one to build and configure it, a different one to query it - which trips up people who expect a single client.
• Key Concepts: Knowledge Base, Data Source, Ingestion Job, vector store, Retrieve, RetrieveAndGenerate.
• When to Use: any application that needs a model to answer from your documents (support docs, internal wikis, product catalogs, contracts) without you hand-building an embeddings and vector-search pipeline.
• Limitations/Trade-offs: you give up some control over chunking internals and prompt construction in exchange for far less code; tight control still means dropping to Retrieve and building your own prompt.
• Related Topics: Bedrock's Converse/InvokeModel APIs, data source ingestion, and RetrieveAndGenerate vs Retrieve-then-prompt.

Foundations

A Knowledge Base is built from three things you provide and one thing AWS runs for you.

You provide: source documents (an S3 bucket is the common case, but web crawlers, Confluence, SharePoint, and Salesforce connectors also exist as Data Source types), an embedding model (a Bedrock embeddings model, referenced by ARN), and a vector store (a place the resulting vectors live - OpenSearch Serverless is the common default, but Aurora PostgreSQL with pgvector, Pinecone, Redis Enterprise Cloud, MongoDB Atlas, and the newer Amazon S3 Vectors are all supported).

AWS runs: the ingestion pipeline that reads your documents, parses them, splits them into chunks, calls the embedding model, and writes vectors into your chosen store. You trigger this with an explicit Ingestion Job rather than it happening automatically on every S3 write.

# --- Python (boto3) ---
import boto3
 
agent = boto3.client("bedrock-agent", region_name="us-east-1")       # control plane
agent_rt = boto3.client("bedrock-agent-runtime", region_name="us-east-1")  # data plane
 
# Once a Knowledge Base exists and has been ingested, querying it is one call:
resp = agent_rt.retrieve(
    knowledgeBaseId="ABCD1234",
    retrievalQuery={"text": "What is our refund policy?"},
)
print(resp["retrievalResults"][0]["content"]["text"])

// --- TypeScript (AWS SDK v3) ---
import { BedrockAgentClient } from "@aws-sdk/client-bedrock-agent";               // control plane
import { BedrockAgentRuntimeClient, RetrieveCommand } from "@aws-sdk/client-bedrock-agent-runtime"; // data plane
 
const agent = new BedrockAgentClient({ region: "us-east-1" });
const agentRt = new BedrockAgentRuntimeClient({ region: "us-east-1" });
 
const resp = await agentRt.send(new RetrieveCommand({
  knowledgeBaseId: "ABCD1234",
  retrievalQuery: { text: "What is our refund policy?" },
}));
console.log(resp.retrievalResults?.[0]?.content?.text);

Notice the import already tells the story: two packages, two clients, two jobs.

Mechanics & Interactions

Control plane - bedrock-agent. This client creates and configures the Knowledge Base itself: CreateKnowledgeBase (name, IAM role, embedding model, vector store config), CreateDataSource (where documents live, plus chunking and parsing strategy), and StartIngestionJob / GetIngestionJob (run and check the pipeline that turns documents into vectors). You touch this client when you stand up or reconfigure a Knowledge Base, and rarely at request time.

Data plane - bedrock-agent-runtime. This client is what your application calls per user request: Retrieve (get ranked passages back, you build the prompt) and RetrieveAndGenerate (get a final, model-generated answer with citations, in one call). This is the client your request handlers import.

Between the two sits the actual pipeline stage sequence:

1. A Data Source describes where documents are and how to read them.
2. Ingestion parses each document (including advanced parsing for PDFs, HTML, and tables), splits it into chunks per your chunking strategy, and calls the embedding model on each chunk.
3. Vectors land in your vector store, tagged with metadata for filtering and with a pointer back to the source location.
4. At query time, Retrieve embeds the query and does a similarity search against the store; RetrieveAndGenerate does that and then feeds the results to a Bedrock foundation model.

The vector store is a plug-in point, not something the Knowledge Base replaces: OpenSearch Serverless is the common default because AWS manages it end to end, but Aurora PostgreSQL (pgvector) suits teams already running Postgres, Pinecone and MongoDB Atlas suit teams standardized on those platforms, and Amazon S3 Vectors is a newer, cost-oriented native option worth comparing on price and latency for large, less latency-sensitive corpora.

Advanced Considerations & Applications

Two design choices ripple through everything else you build on a Knowledge Base.

Chunking and parsing strategy happens once, at ingestion, and is expensive to change later - a new strategy means re-ingesting everything. Get chunk size and overlap right for your document shapes (short FAQ entries vs long contracts) before scaling ingestion.

Retrieve vs RetrieveAndGenerate is a per-request choice, not a Knowledge Base setting. RetrieveAndGenerate is the fastest path to a working RAG answer - one call, managed prompt construction, citations included. Retrieve hands you the raw ranked passages so you can build your own prompt, mix in tool results, or run your own re-ranking - more code, more control.

A newer capability, the agentic retriever, extends this further: instead of a single similarity search, the retrieval step can reformulate a complex question into sub-queries and chain retrieval steps before generation. This is useful for multi-step or comparison-style questions a single vector search answers poorly. It is a fast-evolving 2026 feature - treat exact configuration parameters as something to verify against current bedrock-agent-runtime documentation rather than fixed API surface.

Common Misconceptions

• "Ingestion happens automatically when I add a file to S3." - No, you (or a scheduled job) must call StartIngestionJob; new files sit unindexed until then.
• "One client does everything." - No, bedrock-agent configures the Knowledge Base and bedrock-agent-runtime queries it; mixing them up is the most common first-time error.
• "RetrieveAndGenerate is the only way to use a Knowledge Base." - No, Retrieve gives you the passages directly for a custom prompt or a non-Bedrock generation step.
• "The vector store is baked into the Knowledge Base." - No, it is a configurable backend; several vector databases are supported and swapping is a control-plane, not application, decision.
• "Smart Parsing is required." - No, plain text extraction is the default; advanced parsing is an opt-in setting for documents with complex layout, tables, or images.

FAQs

Related

• Bedrock Knowledge Bases Basics - creating your first Knowledge Base end to end.
• Data Sources & Smart Parsing Ingestion - the ingestion side in depth.
• RetrieveAndGenerate vs Retrieve-Then-Prompt - choosing a query pattern.
• Choosing S3 Vectors vs OpenSearch as a Knowledge Base Store - the vector-store decision.

Stack versions: This page was written for boto3 1.43.x (Python 3.10+) and the AWS SDK for JavaScript v3 (Node.js 18+). Bedrock Knowledge Bases' Agentic Retriever is a fast-evolving 2026 capability - verify exact parameters against current docs.