AI on AWS Best Practices

Q: What IAM permissions should an AI workload have?

Least privilege - only the exact actions you call, such as bedrock:InvokeModel or rekognition:DetectLabels. Avoid broad wildcards, and resolve credentials from IAM roles, not static keys.

Picking an AWS AI service well is a repeatable decision, not a guess. This checklist makes it repeatable.

Because AWS AI spans three layers - purpose-built services, Amazon Bedrock, and Amazon SageMaker AI - the risk is choosing by brand recognition instead of by fit. Each rule below states a habit positively, with a one-line rationale, so you can run it as an audit whenever you start a new AI feature.

Walk it once end to end, then keep it as a review checklist per use case.

How to Use This List

Each item is a rule stated positively, with a short reason.
Groups run roughly in decision order: choose the layer, then optimize cost, then secure and operate.
Check the boxes against your own use case as a quick audit.
Revisit when a new task type, model, or volume level appears.

A - Choose the Right Layer First

Start at the highest layer that fits. Prefer a purpose-built service, then Bedrock, then SageMaker AI - the higher the layer, the less you build.
Match the task shape to the layer. Fixed task with a named service goes to purpose-built; open-ended language or generation goes to Bedrock; custom model goes to SageMaker AI.
Do not default to generative. A tuned task service usually beats a prompted foundation model on a fixed job, for less cost and more consistency.
Chain layers when needed. Extract with a task service, then reason with a foundation model - most real features span layers.

Prefer the tuned task service when one names your job.

# --- Python (boto3) ---
import boto3
 
# Fixed task ("read sentiment") -> purpose-built service, no prompt.
comprehend = boto3.client("comprehend", region_name="us-east-1")
comprehend.detect_sentiment(Text="Fast and friendly.", LanguageCode="en")

// --- TypeScript (AWS SDK v3) ---
import { ComprehendClient, DetectSentimentCommand } from "@aws-sdk/client-comprehend";
 
// Fixed task -> purpose-built service.
const comprehend = new ComprehendClient({ region: "us-east-1" });
await comprehend.send(new DetectSentimentCommand({ Text: "Fast and friendly.", LanguageCode: "en" }));

B - Use Bedrock the Portable Way

Prefer the Converse API. Use Converse/ConverseStream so one request shape works across model providers and switching models is a modelId change.
Enable only the models you use. Request access per model in the Bedrock console, and keep the enabled set minimal.
Cap output length. Set a max-tokens limit so a runaway response cannot dominate the bill.
Stream when latency matters. Use ConverseStream for interactive UX so tokens arrive incrementally instead of all at once.

Keep model calls portable across providers.

# --- Python (boto3) ---
import boto3
 
bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")
bedrock.converse(
    modelId="anthropic.claude-3-5-sonnet-20240620-v1:0",
    messages=[{"role": "user", "content": [{"text": "One-line tip."}]}],
    inferenceConfig={"maxTokens": 200},   # cap output to bound cost
)

// --- TypeScript (AWS SDK v3) ---
import { BedrockRuntimeClient, ConverseCommand } from "@aws-sdk/client-bedrock-runtime";
 
const bedrock = new BedrockRuntimeClient({ region: "us-east-1" });
await bedrock.send(new ConverseCommand({
  modelId: "anthropic.claude-3-5-sonnet-20240620-v1:0",
  messages: [{ role: "user", content: [{ text: "One-line tip." }] }],
  inferenceConfig: { maxTokens: 200 },   // cap output to bound cost
}));

C - Control the Cost Shape

Know the billing shape per service. Token-based (Bedrock on-demand), provisioned throughput (Bedrock), hourly (SageMaker), and per-unit (purpose-built) fail differently at scale.
Right-size the Bedrock mode. Use on-demand for variable traffic; move to provisioned throughput only above a steady-volume threshold.
Tear down idle capacity. SageMaker endpoints and Bedrock provisioned reservations bill even with zero traffic - delete or scale them to zero.
Measure real usage. Read the Bedrock response usage field, tag AI resources, and set AWS Budgets alerts so spend growth surfaces early.

D - Reason About Quality and Reliability

Constrain and validate model output. Ask for a specific format, keep it short, and validate in code - do not trust free text to match a schema.
Expect run-to-run variation. Foundation-model answers can differ between calls; prefer purpose-built services where determinism matters for audits or tests.
Evaluate before adopting a new model. Test a candidate modelId on your task rather than defaulting to the newest one.
Check confidence scores. Purpose-built services return confidence values - use thresholds instead of treating every result as certain.

E - Secure and Govern Access

Grant least-privilege IAM per service. Scope permissions to the exact actions you call (for example bedrock:InvokeModel, textract:AnalyzeDocument), not broad wildcards.
Apply Guardrails on generative output. Use Bedrock Guardrails to enforce content and safety policy on prompts and responses.
Mind data residency and privacy. Know which region processes the data, and avoid sending PII you are not permitted to - use Comprehend PII detection to catch it.
Prefer roles over static keys. Let the SDK resolve credentials from IAM roles on Lambda, ECS, or EC2 rather than embedding keys.

F - Operate and Evolve

Reuse clients across calls. Construct each service client at module scope so warm Lambdas and long-lived servers reuse connections.
Handle access and throttling errors. Catch typed errors - AccessDeniedException often means model access is not enabled; back off on throttling.
Watch the region matrix. Model and service availability varies by region, so confirm your target region supports what you call.
Revisit the choice as volume grows. A layer or pricing mode that fit at low volume may not at scale - re-audit when usage climbs.

FAQs

What is the first question to ask when picking a service?

Does a purpose-built service name my exact task? If yes and volume is high, use it. Otherwise consider Bedrock for open-ended work, and SageMaker AI only when you need a custom model.

Should I default to a foundation model because it is flexible?

No. For fixed, high-volume tasks a tuned purpose-built service is usually cheaper, faster, and more consistent. Reserve foundation models for open-ended or custom-format work.

Why prefer the Bedrock Converse API?

Converse keeps one request shape across model providers, so switching models is a modelId change rather than a rewrite. It also standardizes usage reporting for cost tracking.

How do I keep generative cost under control?

Cap output with a max-tokens setting, keep prompts short, choose an appropriately sized model, read the response usage field, and set AWS Budgets alerts on tagged AI resources.

What is the most common surprise cost?

Idle capacity. SageMaker real-time endpoints and Bedrock provisioned-throughput reservations bill continuously regardless of traffic. Tear down what you are not actively using.

How do I get reliable structured data from a model?

Prompt for a specific short format, then validate it in code. Do not assume the model matches your schema every run; foundation-model output varies between calls.

What IAM permissions should an AI workload have?

Least privilege - only the exact actions you call, such as bedrock:InvokeModel or rekognition:DetectLabels. Avoid broad wildcards, and resolve credentials from IAM roles, not static keys.

What are Bedrock Guardrails for?

They enforce content and safety policy on prompts and responses centrally, so open-ended generation stays within bounds you define. Apply them on user-facing generative features.

Why does region matter for AI services?

Model and service availability, and sometimes pricing, vary by region. Confirm your target region supports the model or service you call before you build against it.

How do I handle a first-call AccessDeniedException on Bedrock?

It usually means model access is not enabled yet. Request access to the model in the Bedrock console, and confirm your IAM policy allows the invoke action for that model.

When should I re-audit my service choice?

Whenever a new task type appears, a new model ships, or volume climbs. A layer or pricing mode that fit at low volume may not at scale, so revisit the decision.

Can I combine purpose-built services and Bedrock?

Yes, and it is the common pattern. Use a task service for the mechanical part (extraction, transcription) and a foundation model for reasoning over the result. Each bills on its own model.

Mapping AWS's AI Surface - the layer model behind rule group A.
Foundation Models vs Purpose-Built AI Services - the choice in group A, in depth.
AWS AI/ML Pricing Models Compared - the cost shapes in group C.
AI on AWS Basics - the first calls these habits build on.

Stack versions: This page was written for boto3 1.43.x (Python 3.10+) and the AWS SDK for JavaScript v3 (Node.js 18+).

AI on AWS Best Practices

Picking an AWS AI service well is a repeatable decision, not a guess. This checklist makes it repeatable.

Walk it once end to end, then keep it as a review checklist per use case.

How to Use This List

Each item is a rule stated positively, with a short reason.
Groups run roughly in decision order: choose the layer, then optimize cost, then secure and operate.
Check the boxes against your own use case as a quick audit.
Revisit when a new task type, model, or volume level appears.

A - Choose the Right Layer First

Start at the highest layer that fits. Prefer a purpose-built service, then Bedrock, then SageMaker AI - the higher the layer, the less you build.
Match the task shape to the layer. Fixed task with a named service goes to purpose-built; open-ended language or generation goes to Bedrock; custom model goes to SageMaker AI.
Do not default to generative. A tuned task service usually beats a prompted foundation model on a fixed job, for less cost and more consistency.
Chain layers when needed. Extract with a task service, then reason with a foundation model - most real features span layers.

Prefer the tuned task service when one names your job.

# --- Python (boto3) ---
import boto3
 
# Fixed task ("read sentiment") -> purpose-built service, no prompt.
comprehend = boto3.client("comprehend", region_name="us-east-1")
comprehend.detect_sentiment(Text="Fast and friendly.", LanguageCode="en")

// --- TypeScript (AWS SDK v3) ---
import { ComprehendClient, DetectSentimentCommand } from "@aws-sdk/client-comprehend";
 
// Fixed task -> purpose-built service.
const comprehend = new ComprehendClient({ region: "us-east-1" });
await comprehend.send(new DetectSentimentCommand({ Text: "Fast and friendly.", LanguageCode: "en" }));

B - Use Bedrock the Portable Way

Prefer the Converse API. Use Converse/ConverseStream so one request shape works across model providers and switching models is a modelId change.
Enable only the models you use. Request access per model in the Bedrock console, and keep the enabled set minimal.
Cap output length. Set a max-tokens limit so a runaway response cannot dominate the bill.
Stream when latency matters. Use ConverseStream for interactive UX so tokens arrive incrementally instead of all at once.

Keep model calls portable across providers.

# --- Python (boto3) ---
import boto3
 
bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")
bedrock.converse(
    modelId="anthropic.claude-3-5-sonnet-20240620-v1:0",
    messages=[{"role": "user", "content": [{"text": "One-line tip."}]}],
    inferenceConfig={"maxTokens": 200},   # cap output to bound cost
)

// --- TypeScript (AWS SDK v3) ---
import { BedrockRuntimeClient, ConverseCommand } from "@aws-sdk/client-bedrock-runtime";
 
const bedrock = new BedrockRuntimeClient({ region: "us-east-1" });
await bedrock.send(new ConverseCommand({
  modelId: "anthropic.claude-3-5-sonnet-20240620-v1:0",
  messages: [{ role: "user", content: [{ text: "One-line tip." }] }],
  inferenceConfig: { maxTokens: 200 },   // cap output to bound cost
}));

C - Control the Cost Shape

Know the billing shape per service. Token-based (Bedrock on-demand), provisioned throughput (Bedrock), hourly (SageMaker), and per-unit (purpose-built) fail differently at scale.
Right-size the Bedrock mode. Use on-demand for variable traffic; move to provisioned throughput only above a steady-volume threshold.
Tear down idle capacity. SageMaker endpoints and Bedrock provisioned reservations bill even with zero traffic - delete or scale them to zero.
Measure real usage. Read the Bedrock response usage field, tag AI resources, and set AWS Budgets alerts so spend growth surfaces early.

D - Reason About Quality and Reliability

Constrain and validate model output. Ask for a specific format, keep it short, and validate in code - do not trust free text to match a schema.
Expect run-to-run variation. Foundation-model answers can differ between calls; prefer purpose-built services where determinism matters for audits or tests.
Evaluate before adopting a new model. Test a candidate modelId on your task rather than defaulting to the newest one.
Check confidence scores. Purpose-built services return confidence values - use thresholds instead of treating every result as certain.

E - Secure and Govern Access

Grant least-privilege IAM per service. Scope permissions to the exact actions you call (for example bedrock:InvokeModel, textract:AnalyzeDocument), not broad wildcards.
Apply Guardrails on generative output. Use Bedrock Guardrails to enforce content and safety policy on prompts and responses.
Mind data residency and privacy. Know which region processes the data, and avoid sending PII you are not permitted to - use Comprehend PII detection to catch it.
Prefer roles over static keys. Let the SDK resolve credentials from IAM roles on Lambda, ECS, or EC2 rather than embedding keys.

F - Operate and Evolve

Reuse clients across calls. Construct each service client at module scope so warm Lambdas and long-lived servers reuse connections.
Handle access and throttling errors. Catch typed errors - AccessDeniedException often means model access is not enabled; back off on throttling.
Watch the region matrix. Model and service availability varies by region, so confirm your target region supports what you call.
Revisit the choice as volume grows. A layer or pricing mode that fit at low volume may not at scale - re-audit when usage climbs.

FAQs

What is the first question to ask when picking a service?

Does a purpose-built service name my exact task? If yes and volume is high, use it. Otherwise consider Bedrock for open-ended work, and SageMaker AI only when you need a custom model.

Should I default to a foundation model because it is flexible?

No. For fixed, high-volume tasks a tuned purpose-built service is usually cheaper, faster, and more consistent. Reserve foundation models for open-ended or custom-format work.

Why prefer the Bedrock Converse API?

Converse keeps one request shape across model providers, so switching models is a modelId change rather than a rewrite. It also standardizes usage reporting for cost tracking.

How do I keep generative cost under control?

Cap output with a max-tokens setting, keep prompts short, choose an appropriately sized model, read the response usage field, and set AWS Budgets alerts on tagged AI resources.

What is the most common surprise cost?

Idle capacity. SageMaker real-time endpoints and Bedrock provisioned-throughput reservations bill continuously regardless of traffic. Tear down what you are not actively using.

How do I get reliable structured data from a model?

Prompt for a specific short format, then validate it in code. Do not assume the model matches your schema every run; foundation-model output varies between calls.

What IAM permissions should an AI workload have?

Least privilege - only the exact actions you call, such as bedrock:InvokeModel or rekognition:DetectLabels. Avoid broad wildcards, and resolve credentials from IAM roles, not static keys.

What are Bedrock Guardrails for?

They enforce content and safety policy on prompts and responses centrally, so open-ended generation stays within bounds you define. Apply them on user-facing generative features.

Why does region matter for AI services?

Model and service availability, and sometimes pricing, vary by region. Confirm your target region supports the model or service you call before you build against it.

How do I handle a first-call AccessDeniedException on Bedrock?

It usually means model access is not enabled yet. Request access to the model in the Bedrock console, and confirm your IAM policy allows the invoke action for that model.

When should I re-audit my service choice?

Whenever a new task type appears, a new model ships, or volume climbs. A layer or pricing mode that fit at low volume may not at scale, so revisit the decision.

Can I combine purpose-built services and Bedrock?

Yes, and it is the common pattern. Use a task service for the mechanical part (extraction, transcription) and a foundation model for reasoning over the result. Each bills on its own model.

Mapping AWS's AI Surface - the layer model behind rule group A.
Foundation Models vs Purpose-Built AI Services - the choice in group A, in depth.
AWS AI/ML Pricing Models Compared - the cost shapes in group C.
AI on AWS Basics - the first calls these habits build on.

Stack versions: This page was written for boto3 1.43.x (Python 3.10+) and the AWS SDK for JavaScript v3 (Node.js 18+).

AI on AWS Best Practices

Search Documentation

AI on AWS Best Practices