Anatomy of an AWS API Request

Under every SDK call is a plain HTTPS request.

This page opens up that request: the endpoint it targets, the headers it carries, and the SigV4 signature that proves who you are.

Summary

An AWS API call is an HTTPS request to a service endpoint, usually POST or GET, with a JSON, XML, or query-string body.

What makes it an AWS request is the signature. AWS uses Signature Version 4 (SigV4) to authenticate the caller and verify the request was not altered.

The SDK builds the canonical request, computes the signature from your secret key, and attaches it in the Authorization header.

Your secret key never travels over the wire. Only a derived signature does, so an intercepted request cannot reveal your credentials.

Every response comes back with a request id in its headers, which is the single most useful value to log when something fails.

Recipe

You rarely sign by hand. But you can generate a presigned URL, which is a signed request you can hand to someone else.

# --- Python (boto3) ---
import boto3
 
s3 = boto3.client("s3")
url = s3.generate_presigned_url(
    "get_object",
    Params={"Bucket": "my-bucket", "Key": "report.pdf"},
    ExpiresIn=3600,   # signature valid for one hour
)
print(url)

// --- TypeScript (AWS SDK v3) ---
import { S3Client, GetObjectCommand } from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
 
const s3 = new S3Client({});
const url = await getSignedUrl(
  s3,
  new GetObjectCommand({ Bucket: "my-bucket", Key: "report.pdf" }),
  { expiresIn: 3600 },
);
console.log(url);

When to reach for this:

• You want to give a browser or third party temporary access without sharing credentials.
• You need the SigV4 signature encoded into a URL rather than headers.
• You want a time-limited grant (the ExpiresIn / expiresIn seconds).
• You are debugging and want to see a fully signed request you can inspect.

Working Example

This inspects the exact HTTP request boto3 would send, without actually sending it. It is the clearest way to see the anatomy.

# --- Python (boto3) ---
import boto3
 
s3 = boto3.client("s3", region_name="us-east-1")
 
# Register a hook that fires just before the signed request goes out.
def show(request, **kwargs):
    print(request.method, request.url)
    for name, value in request.headers.items():
        print(f"{name}: {value}")
 
s3.meta.events.register("before-send.s3.ListBuckets", show)
s3.list_buckets()   # prints method, URL, and signed headers

// --- TypeScript (AWS SDK v3) ---
import { S3Client, ListBucketsCommand } from "@aws-sdk/client-s3";
 
const s3 = new S3Client({ region: "us-east-1" });
 
// Middleware that logs the fully built, signed HTTP request.
s3.middlewareStack.add(
  (next) => async (args) => {
    const req = args.request as { method: string; hostname: string; headers: Record<string, string> };
    console.log(req.method, req.hostname);
    for (const [name, value] of Object.entries(req.headers)) console.log(`${name}: ${value}`);
    return next(args);
  },
  { step: "finalizeRequest" },
);
 
await s3.send(new ListBucketsCommand({}));

What this demonstrates:

• boto3 exposes lifecycle events; before-send fires after signing, so you see the real headers.
• SDK v3 uses a middleware stack; the finalizeRequest step runs after the signer.
• You will see Host, X-Amz-Date, Authorization, and usually X-Amz-Content-Sha256.
• The Authorization header contains the signature but never your secret key.

Deep Dive

How It Works

• The SDK resolves the endpoint from service + region, then builds a canonical request: method, path, query, headers, and a hash of the body.
• It derives a signing key from your secret access key, the date, region, and service.
• It signs the canonical request with that key using HMAC-SHA256, producing the signature.
• It attaches the signature, your access key id, and the signed-header list in the Authorization header.
• AWS recomputes the signature server-side; if it matches, the caller is authenticated and the body is intact.

Key Headers

Why Signing Matters

• The signature proves identity without sending the secret key over the network.
• It covers the timestamp, so a captured request expires (SigV4 requests are valid for a limited window).
• It hashes the body, so tampering in transit invalidates the signature.
• Temporary credentials add a session token header that must also be present and valid.

Finding the Request Id

• Every response includes a request id in the x-amz-request-id (and often x-amz-id-2) headers.
• In boto3 it is at response["ResponseMetadata"]["RequestId"].
• In SDK v3 it is at response.$metadata.requestId.
• Log it on both success and failure; AWS support needs it to trace a specific call.

Gotchas

• Clock skew breaks signing - if the machine's clock is off by more than the allowed window, AWS rejects the signature. Fix: sync the system clock with NTP.
• Reusing a presigned URL after expiry - the embedded signature is time-limited. Fix: set a realistic ExpiresIn and regenerate when it lapses.
• Missing session token - temporary credentials require X-Amz-Security-Token; omitting it fails auth. Fix: let the SDK resolve full credentials rather than passing only key and secret.
• Modifying a signed request - changing the URL, headers, or body after signing invalidates the signature. Fix: set all parameters before the call; do not mutate the outgoing request.
• Not logging the request id - failures are hard to trace without it. Fix: log RequestId / $metadata.requestId on every error.
• Assuming HTTP is fine - AWS endpoints require HTTPS; SigV4 assumes TLS in transit. Fix: never override the endpoint to plain HTTP except against a local test double.

Alternatives

FAQs

Related

• What an AWS SDK Actually Does - where signing sits in the SDK pipeline.
• Service Endpoints, Partitions & Dual-Stack/FIPS Variants - how the endpoint in the Host header is chosen.
• How AWS Bills SDK Calls - each signed request is a billable request.
• AWS SDK Fundamentals Basics - making your first calls.

Stack versions: This page was written for boto3 1.43.x (Python 3.10+) and the AWS SDK for JavaScript v3 (Node.js 18+). Y29kZWd1aWRlcy5pb3xjZ2lvMTE1NXwyMDI2MDc=