SageMaker Deployment Basics

Six examples to get a trained model serving real-time predictions - four basic and two intermediate.

Each example shows the same operation in Python (boto3) and TypeScript (AWS SDK v3), deploying a model artifact produced by a prior training job to a single-instance real-time endpoint.

Prerequisites

• Python: pip install boto3 (boto3 1.43.x, Python 3.10+). TypeScript: npm install @aws-sdk/client-sagemaker @aws-sdk/client-sagemaker-runtime (Node.js 18+).
• A trained model artifact already in S3 (a model.tar.gz, typically ModelArtifacts.S3ModelArtifacts from a completed training job).
• An IAM role that SageMaker can assume to read that artifact and pull the serving container image (a "SageMaker execution role").
• Credentials configured via aws configure, environment variables, or an IAM role - the SDK resolves them automatically.

Quick Install

# --- Python (boto3) ---
import boto3
 
sm = boto3.client("sagemaker", region_name="us-east-1")
runtime = boto3.client("sagemaker-runtime", region_name="us-east-1")

// --- TypeScript (AWS SDK v3) ---
import { SageMakerClient } from "@aws-sdk/client-sagemaker";
import { SageMakerRuntimeClient } from "@aws-sdk/client-sagemaker-runtime";
 
const sm = new SageMakerClient({ region: "us-east-1" });
const runtime = new SageMakerRuntimeClient({ region: "us-east-1" });

Basic Examples

1. Register the Model

CreateModel points at the artifact and the container image that will serve it. It registers a resource - nothing runs yet.

# --- Python (boto3) ---
sm.create_model(
    ModelName="xgboost-basics-model",
    PrimaryContainer={
        "Image": "683313688378.dkr.ecr.us-east-1.amazonaws.com/xgboost:1.7-1",
        "ModelDataUrl": "s3://my-ml-bucket/training-output/xgboost-basics-001/output/model.tar.gz",
    },
    ExecutionRoleArn="arn:aws:iam::111122223333:role/SageMakerExecutionRole",
)

// --- TypeScript (AWS SDK v3) ---
import { CreateModelCommand } from "@aws-sdk/client-sagemaker";
 
await sm.send(new CreateModelCommand({
  ModelName: "xgboost-basics-model",
  PrimaryContainer: {
    Image: "683313688378.dkr.ecr.us-east-1.amazonaws.com/xgboost:1.7-1",
    ModelDataUrl: "s3://my-ml-bucket/training-output/xgboost-basics-001/output/model.tar.gz",
  },
  ExecutionRoleArn: "arn:aws:iam::111122223333:role/SageMakerExecutionRole",
}));

• ModelDataUrl is exactly the S3 artifact a training job produced - reuse it as-is, no repackaging.
• Image can be the same built-in algorithm image used for training, or a distinct inference-only container.
• A model resource costs nothing by itself; it only matters once an endpoint references it.
• ModelName must be unique per account/region - reusing one before deleting the old model fails the call.

Related: SageMaker's Inference Deployment Options.

2. Configure the Endpoint

CreateEndpointConfig's ProductionVariants picks the instance type, count, and which model to run.

# --- Python (boto3) ---
sm.create_endpoint_config(
    EndpointConfigName="xgboost-basics-config",
    ProductionVariants=[{
        "VariantName": "AllTraffic",
        "ModelName": "xgboost-basics-model",
        "InstanceType": "ml.m5.large",
        "InitialInstanceCount": 1,
    }],
)

// --- TypeScript (AWS SDK v3) ---
import { CreateEndpointConfigCommand } from "@aws-sdk/client-sagemaker";
 
await sm.send(new CreateEndpointConfigCommand({
  EndpointConfigName: "xgboost-basics-config",
  ProductionVariants: [{
    VariantName: "AllTraffic",
    ModelName: "xgboost-basics-model",
    InstanceType: "ml.m5.large",
    InitialInstanceCount: 1,
  }],
}));

• ProductionVariants is a list - a single-model endpoint just has one entry, named anything (AllTraffic is a common convention).
• InstanceType/InitialInstanceCount here mark this as a real-time endpoint; other pages in this section swap this variant shape for serverless, async, or multi-model configs.
• An endpoint config is a reusable, immutable blueprint - creating a new one doesn't affect a running endpoint until you update it to point at the new config.
• No compute is provisioned yet at this step either; that happens on CreateEndpoint.

3. Create the Endpoint

CreateEndpoint provisions the instances described in the config and starts them serving.

# --- Python (boto3) ---
sm.create_endpoint(
    EndpointName="xgboost-basics-endpoint",
    EndpointConfigName="xgboost-basics-config",
)

// --- TypeScript (AWS SDK v3) ---
import { CreateEndpointCommand } from "@aws-sdk/client-sagemaker";
 
await sm.send(new CreateEndpointCommand({
  EndpointName: "xgboost-basics-endpoint",
  EndpointConfigName: "xgboost-basics-config",
}));

• The call returns immediately with an EndpointArn; provisioning instances and pulling the container image happens asynchronously.
• Billing for a real-time endpoint starts once its instances are up, and continues hourly until the endpoint is deleted.
• EndpointName is what callers use later to invoke it, and what future updates (UpdateEndpoint) reference.
• Standing up a new endpoint typically takes several minutes - it is not an instant operation.

Related: Real-Time Endpoints & Auto Scaling.

4. Check Endpoint Status

DescribeEndpoint reports EndpointStatus - Creating, InService, Updating, Failed, or Deleting.

# --- Python (boto3) ---
desc = sm.describe_endpoint(EndpointName="xgboost-basics-endpoint")
print("status:", desc["EndpointStatus"])
if desc["EndpointStatus"] == "Failed":
    print("reason:", desc["FailureReason"])

// --- TypeScript (AWS SDK v3) ---
import { DescribeEndpointCommand } from "@aws-sdk/client-sagemaker";
 
const desc = await sm.send(new DescribeEndpointCommand({ EndpointName: "xgboost-basics-endpoint" }));
console.log("status:", desc.EndpointStatus);
if (desc.EndpointStatus === "Failed") {
  console.log("reason:", desc.FailureReason);
}

• Creating covers everything from instance provisioning through the container's health check passing.
• FailureReason is a plain-text summary - deeper diagnostics live in CloudWatch Logs under /aws/sagemaker/Endpoints.
• Only an InService endpoint should receive production traffic; earlier statuses will reject or fail invocations.
• A single describe call is cheap enough to poll on an interval, but prefer the waiter below over a hand-rolled loop.

Intermediate Examples

5. Wait for the Endpoint to Come Up

Instead of hand-rolled polling, use the endpoint_in_service waiter (boto3) or its SDK v3 equivalent.

# --- Python (boto3) ---
waiter = sm.get_waiter("endpoint_in_service")
waiter.wait(
    EndpointName="xgboost-basics-endpoint",
    WaiterConfig={"Delay": 30, "MaxAttempts": 60},   # up to 30 minutes
)
print("endpoint is in service")

// --- TypeScript (AWS SDK v3) ---
import { waitUntilEndpointInService } from "@aws-sdk/client-sagemaker";
 
await waitUntilEndpointInService(
  { client: sm, maxWaitTime: 1800 },
  { EndpointName: "xgboost-basics-endpoint" },
);
console.log("endpoint is in service");

• The waiter polls DescribeEndpoint on your behalf and returns once the status reaches a terminal state.
• It does not throw on Failed in every SDK - check EndpointStatus explicitly afterward rather than assuming success.
• First-time creation is typically the slowest path; later config updates on an existing endpoint are often quicker.
• Keep the timeout generous for larger instance types or container images with a slow cold start on first pull.

6. Invoke the Endpoint

Predictions go through sagemaker-runtime, a separate client from the one used to create the endpoint.

# --- Python (boto3) ---
import json
 
response = runtime.invoke_endpoint(
    EndpointName="xgboost-basics-endpoint",
    ContentType="text/csv",
    Body="1.5,0.2,3.1,0.4",
)
result = response["Body"].read().decode("utf-8")
print("prediction:", result)

// --- TypeScript (AWS SDK v3) ---
import { InvokeEndpointCommand } from "@aws-sdk/client-sagemaker-runtime";
 
const response = await runtime.send(new InvokeEndpointCommand({
  EndpointName: "xgboost-basics-endpoint",
  ContentType: "text/csv",
  Body: Buffer.from("1.5,0.2,3.1,0.4"),
}));
const result = Buffer.from(response.Body as Uint8Array).toString("utf-8");
console.log("prediction:", result);

• ContentType must match what the container's inference code expects to parse - CSV, JSON, or another format.
• Body is raw bytes either way; the SDK does not serialize it for you beyond what you pass in.
• The response Body is also raw bytes and needs decoding on both sides before it's usable as text or JSON.
• This is the one call in the whole flow billed per-request in addition to the endpoint's hourly instance cost - everything before it is control-plane and free.

Related: SageMaker Deployment via SDK Best Practices.

Stack versions: This page was written for boto3 1.43.x (Python 3.10+) and the AWS SDK for JavaScript v3 (Node.js 18+).