Bedrock Knowledge Bases Best Practices

Chunking strategy, metadata filtering, and retrieval-quality evaluation.

This is the checklist to run before and after putting a Bedrock Knowledge Base into production. Each item is a rule stated positively with a one-line rationale, grouped from setup outward through ingestion, retrieval, cost, and security. Work top to bottom - early groups keep your Knowledge Base structurally sound; later ones keep it accurate, cheap, and safe.

How to Use This List

• Treat each box as an audit question against your own Knowledge Base integration.
• Groups run roughly in order of foundation: get chunking and storage right first, then ingestion hygiene, retrieval quality, cost, and security.
• Most rules apply regardless of which vector store or generation model you chose.
• Revisit the list whenever you add a Data Source, change document types, or adopt the agentic retriever.

A - Setup & Chunking Strategy

• Choose chunking strategy deliberately, before scale. FIXED_SIZE is the safe default; changing strategy later means re-ingesting the whole Data Source, not a config toggle.
• Set a modest chunk overlap. Zero overlap means an answer spanning a chunk boundary can be split and missed; 10-20% overlap is a reasonable default.
• Reserve Smart Parsing for documents that need it. BEDROCK_FOUNDATION_MODEL parsing costs an extra model call per document - use it for PDFs/tables/complex layout, not already-clean plain text.
• Pick the vector store for your traffic shape, not by default alone. OpenSearch Serverless suits latency-sensitive high-QPS retrieval; S3 Vectors suits large or cost-conscious, less latency-critical corpora.
• Scope the service role narrowly. The roleArn passed to CreateKnowledgeBase should read only the intended Data Sources and invoke only the intended embedding model.

B - Ingestion Hygiene

• Guard against overlapping ingestion jobs. Only one StartIngestionJob runs per Data Source at a time; check ListIngestionJobs status before starting another in an automated scheduler.
• Always inspect statistics, not just job status. A job can report COMPLETE while numberOfDocumentsFailed is nonzero - alert on that field, not just overall success.
• Trigger ingestion on real change events. Wire StartIngestionJob to S3 write events (via EventBridge) or a pipeline's last step rather than relying on someone to remember.
• Account for deletion lag. A removed source document's vectors persist until the next ingestion job reconciles it - sync promptly if staleness has compliance implications.
• Re-ingest fully after an ingestion-config change. Incremental detection can skip unchanged documents even after you change chunking or parsing settings - force a full re-ingest when the pipeline configuration itself changes.

Configure the control-plane client with sane retries, since ingestion and creation calls can be throttled under load.

# --- Python (boto3) ---
import boto3
from botocore.config import Config
 
cfg = Config(retries={"max_attempts": 5, "mode": "adaptive"})
agent = boto3.client("bedrock-agent", region_name="us-east-1", config=cfg)
agent_rt = boto3.client("bedrock-agent-runtime", region_name="us-east-1", config=cfg)

// --- TypeScript (AWS SDK v3) ---
import { BedrockAgentClient } from "@aws-sdk/client-bedrock-agent";
import { BedrockAgentRuntimeClient } from "@aws-sdk/client-bedrock-agent-runtime";
 
const agent = new BedrockAgentClient({ region: "us-east-1", maxAttempts: 5 });
const agentRt = new BedrockAgentRuntimeClient({ region: "us-east-1", maxAttempts: 5 });

C - Retrieval Quality

• Default to RetrieveAndGenerate, drop to Retrieve-then-prompt with a reason. Only build your own prompt when you need custom structure, mixed-in context, or a non-Bedrock generation step.
• Use metadata filtering to narrow results. Tag chunks with metadata (product, region, access tier) at ingestion and filter on it at query time instead of relying on similarity alone.
• Evaluate retrieval quality with real queries. Run your actual user questions against the Knowledge Base and check whether the right passages come back before trusting it in production.
• Tune numberOfResults from evals, not guesswork. Too few misses context; too many dilutes the generation prompt and raises token cost.
• Reserve the agentic retriever for genuinely multi-step questions. It adds latency and cost from extra reformulation and retrieval calls - do not enable it for every query by default.
• Always inspect citations. Even a "successful" RetrieveAndGenerate call can under-cover a multi-part question - check that citations back every part of the answer.

D - Cost Monitoring

• Log token usage on every generation call. RetrieveAndGenerate and Converse both report usage - emit it as a metric to catch cost regressions early.
• Right-size the generation model. A top-tier model for straightforward Q&A over your docs is often wasted spend; benchmark a smaller tier first.
• Budget for Smart Parsing separately from embedding cost. It is an additional per-document model call at ingestion, not part of the embedding bill.
• Watch vector-store cost against traffic shape. OpenSearch Serverless bills a capacity floor even near-idle; S3 Vectors follows pay-per-use - mismatch here quietly overpays.
• Trim retrieved context before generation when it is clearly redundant. Duplicate or near-duplicate passages inflate the generation prompt's input tokens for no retrieval-quality benefit.

E - Security & IAM

• Separate control-plane and data-plane permissions. Grant bedrock-agent actions (CreateKnowledgeBase, CreateDataSource, StartIngestionJob) only to admin/setup roles; grant bedrock-agent-runtime actions (Retrieve, RetrieveAndGenerate) to request-handling roles.
• Scope the Knowledge Base service role tightly. It should read only the Data Sources it owns and call only the embedding (and, if used, parsing) model it needs - not a broad S3 or Bedrock grant.
• Avoid logging sensitive retrieved content. Retrieved passages can carry the same sensitivity as the source documents; log request IDs and token counts by default, not raw passage text, without a clear data-handling policy.
• Use metadata-based access control for multi-tenant corpora. Filter retrieval by a tenant/user metadata field rather than relying on separate Knowledge Bases per tenant unless isolation truly requires it.
• Capture request IDs on failures. Log $metadata.requestId (v3) or the boto3 response metadata for both control-plane and data-plane calls so AWS support can trace issues.

FAQs

Related

• Data Sources & Smart Parsing Ingestion - the chunking and parsing rules in group A.
• Sync Jobs & Incremental Ingestion - the ingestion hygiene rules in group B.
• RetrieveAndGenerate vs Retrieve-Then-Prompt - the retrieval-pattern rules in group C.
• Choosing S3 Vectors vs OpenSearch as a Knowledge Base Store - the cost trade-offs in group D.

Stack versions: This page was written for boto3 1.43.x (Python 3.10+) and the AWS SDK for JavaScript v3 (Node.js 18+). Bedrock Knowledge Bases' Agentic Retriever is a fast-evolving 2026 capability - verify exact parameters against current docs.