S3 Advanced Basics

The Core section showed one bucket, one object, one put, one get. Real buckets hold thousands or millions of objects, and you rarely touch them one at a time.

This page covers the everyday bulk operations that sit just above put and get: listing an entire prefix past the 1,000-key page limit, deleting objects in batches, reading metadata without downloading bodies, copying server-side, and emptying a prefix. Every example mirrors between boto3 and the AWS SDK v3.

Prerequisites

• Python: pip install boto3 (boto3 1.43.x, Python 3.10+). TypeScript: npm install @aws-sdk/client-s3 (Node.js 18+).
• Credentials configured via aws configure, environment variables, or an IAM role - the SDK resolves them automatically.
• An existing bucket with a few objects to list and delete. These examples assume a bucket named my-bucket in us-east-1.

Basic Examples

1. List Every Object Under a Prefix (Paginated)

ListObjectsV2 returns at most 1,000 keys per response. A paginator walks all pages for you so you never miss objects.

# --- Python (boto3) ---
import boto3
 
s3 = boto3.client("s3")
paginator = s3.get_paginator("list_objects_v2")
for page in paginator.paginate(Bucket="my-bucket", Prefix="logs/"):
    for obj in page.get("Contents", []):
        print(obj["Key"], obj["Size"])

// --- TypeScript (AWS SDK v3) ---
import { S3Client, paginateListObjectsV2 } from "@aws-sdk/client-s3";
 
const s3 = new S3Client({});
for await (const page of paginateListObjectsV2({ client: s3 }, { Bucket: "my-bucket", Prefix: "logs/" })) {
  for (const obj of page.Contents ?? []) console.log(obj.Key, obj.Size);
}

• Paginators hide the ContinuationToken loop, so you get every key without manual bookkeeping.
• Prefix filters server-side; only keys starting with that string are returned.
• Contents is absent (not empty) when a page matches nothing, so default it to [].
• Each entry carries Key, Size, LastModified, ETag, and StorageClass.

Related: Amazon S3 via SDK: Buckets & Objects - the single-object put/get basics.

2. Delete Many Objects in One Request

DeleteObjects removes up to 1,000 keys per call. That is one request and one round trip instead of a thousand.

# --- Python (boto3) ---
import boto3
 
s3 = boto3.client("s3")
resp = s3.delete_objects(
    Bucket="my-bucket",
    Delete={"Objects": [{"Key": "logs/a.txt"}, {"Key": "logs/b.txt"}]},
)
print("deleted:", [d["Key"] for d in resp.get("Deleted", [])])

// --- TypeScript (AWS SDK v3) ---
import { S3Client, DeleteObjectsCommand } from "@aws-sdk/client-s3";
 
const s3 = new S3Client({});
const resp = await s3.send(new DeleteObjectsCommand({
  Bucket: "my-bucket",
  Delete: { Objects: [{ Key: "logs/a.txt" }, { Key: "logs/b.txt" }] },
}));
console.log("deleted:", (resp.Deleted ?? []).map((d) => d.Key));

• The Delete.Objects list caps at 1,000 keys per request - split larger sets into chunks.
• The response separates Deleted from Errors, so check both after each call.
• Batch deletes are dramatically cheaper than per-object DeleteObject at scale.
• On a versioned bucket, omitting VersionId adds a delete marker rather than erasing history.

3. Read Metadata Without Downloading (HeadObject)

HeadObject returns size, content type, and metadata but no body - perfect for existence checks and inspection.

# --- Python (boto3) ---
import boto3
 
s3 = boto3.client("s3")
head = s3.head_object(Bucket="my-bucket", Key="logs/a.txt")
print(head["ContentLength"], head["ContentType"], head.get("Metadata", {}))

// --- TypeScript (AWS SDK v3) ---
import { S3Client, HeadObjectCommand } from "@aws-sdk/client-s3";
 
const s3 = new S3Client({});
const head = await s3.send(new HeadObjectCommand({ Bucket: "my-bucket", Key: "logs/a.txt" }));
console.log(head.ContentLength, head.ContentType, head.Metadata);

• HeadObject transfers only headers, so it is fast and cheap for existence checks.
• A missing object raises 404/NotFound; catch it to test whether a key exists.
• Metadata holds your custom x-amz-meta-* headers as a plain map.
• Use it before a download to decide on size, type, or caching without pulling bytes.

Intermediate Examples

4. Copy or Rename an Object Server-Side

S3 has no rename. You copy to the new key, then delete the old one. The copy happens inside S3, so bytes never pass through your process.

# --- Python (boto3) ---
import boto3
 
s3 = boto3.client("s3")
s3.copy_object(
    Bucket="my-bucket",
    CopySource={"Bucket": "my-bucket", "Key": "logs/a.txt"},
    Key="archive/a.txt",
)
s3.delete_object(Bucket="my-bucket", Key="logs/a.txt")  # complete the "rename"

// --- TypeScript (AWS SDK v3) ---
import { S3Client, CopyObjectCommand, DeleteObjectCommand } from "@aws-sdk/client-s3";
 
const s3 = new S3Client({});
await s3.send(new CopyObjectCommand({
  Bucket: "my-bucket",
  CopySource: "my-bucket/logs/a.txt",   // "bucket/key", URL-encode special chars
  Key: "archive/a.txt",
}));
await s3.send(new DeleteObjectCommand({ Bucket: "my-bucket", Key: "logs/a.txt" }));

• boto3 takes CopySource as a dict; SDK v3 takes it as a "bucket/key" string.
• A rename is a copy plus a delete - there is no atomic move operation.
• Server-side copy avoids egress and is fast, but objects over 5 GB need multipart copy.
• You can change storage class or metadata during the copy with extra parameters.

5. Empty a Whole Prefix (List + Batch Delete)

Combine the paginator and batch delete to clean out a prefix - the safe way to empty a bucket before deleting it.

# --- Python (boto3) ---
import boto3
 
s3 = boto3.client("s3")
paginator = s3.get_paginator("list_objects_v2")
for page in paginator.paginate(Bucket="my-bucket", Prefix="tmp/"):
    keys = [{"Key": o["Key"]} for o in page.get("Contents", [])]
    if keys:
        s3.delete_objects(Bucket="my-bucket", Delete={"Objects": keys})

// --- TypeScript (AWS SDK v3) ---
import { S3Client, paginateListObjectsV2, DeleteObjectsCommand } from "@aws-sdk/client-s3";
 
const s3 = new S3Client({});
for await (const page of paginateListObjectsV2({ client: s3 }, { Bucket: "my-bucket", Prefix: "tmp/" })) {
  const keys = (page.Contents ?? []).map((o) => ({ Key: o.Key! }));
  if (keys.length) await s3.send(new DeleteObjectsCommand({ Bucket: "my-bucket", Delete: { Objects: keys } }));
}

• Each ListObjectsV2 page already returns up to 1,000 keys, which is exactly one batch delete's worth.
• Guard against empty pages - DeleteObjects rejects an empty Objects list.
• On a versioned bucket, this leaves delete markers; use list_object_versions to erase every version.
• This is the standard "empty then delete" pattern for tearing a bucket down.

Stack versions: This page was written for boto3 1.43.x (Python 3.10+) and the AWS SDK for JavaScript v3 (Node.js 18+).