A paginator is the SDK's built-in loop for following that token to the end. This page shows the automatic paginators in both SDKs, how to control page size, and when to fall back to manual token handling.
## Recipe
> Ask the client for a paginator, then iterate. No token bookkeeping.
<CodeGroup labels="Python (boto3), TypeScript (AWS SDK v3)">
```python
# --- Python (boto3) ---
import boto3
s3 = boto3.client("s3")
paginator = s3.get_paginator("list_objects_v2")
for page in paginator.paginate(Bucket="my-bucket"):
for obj in page.get("Contents", []):
print(obj["Key"])
```
```typescript
// --- TypeScript (AWS SDK v3) ---
import { S3Client, paginateListObjectsV2 } from "@aws-sdk/client-s3";
const s3 = new S3Client({});
for await (const page of paginateListObjectsV2({ client: s3 }, { Bucket: "my-bucket" })) {
for (const obj of page.Contents ?? []) console.log(obj.Key);
}
```
</CodeGroup>
**When to reach for this:**
- The operation name starts with `List`, `Describe`, `Scan`, `Query`, or `Get*` and can return a token.
- You want every result, not just the first page.
- You do not want to manage `NextToken` / `ContinuationToken` by hand.
- You want throttling and retries handled per page automatically.
## Working Example
A realistic job: list every object under a prefix, cap the page size to bound capacity, and stop after a maximum.
<CodeGroup labels="Python (boto3), TypeScript (AWS SDK v3)">
```python
# --- Python (boto3) ---
import boto3
s3 = boto3.client("s3")
paginator = s3.get_paginator("list_objects_v2")
pages = paginator.paginate(
Bucket="my-bucket",
Prefix="logs/2026/",
PaginationConfig={"PageSize": 200, "MaxItems": 1000},
)
count = 0
for page in pages:
for obj in page.get("Contents", []):
count += 1
print(obj["Key"], obj["Size"])
print(f"listed {count} objects")
```
```typescript
// --- TypeScript (AWS SDK v3) ---
import { S3Client, paginateListObjectsV2 } from "@aws-sdk/client-s3";
const s3 = new S3Client({});
const pages = paginateListObjectsV2(
{ client: s3, pageSize: 200 },
{ Bucket: "my-bucket", Prefix: "logs/2026/" },
);
let count = 0;
for await (const page of pages) {
for (const obj of page.Contents ?? []) {
count += 1;
console.log(obj.Key, obj.Size);
if (count >= 1000) break;
}
if (count >= 1000) break;
}
console.log(`listed ${count} objects`);
```
</CodeGroup>
**What this demonstrates:**
- `Prefix` filters server-side, so you fetch only matching keys.
- boto3's `PaginationConfig` sets both per-request `PageSize` and an overall `MaxItems` cap.
- SDK v3 sets `pageSize` in the config argument but has no built-in overall cap, so you break out manually.
- Both process items as they arrive, keeping memory flat regardless of total size.
## Deep Dive
### How the Paginator Works
- boto3's `get_paginator(name)` looks up the operation's pagination model (input token, output token, result key, page-size field) from the service definition.
- `.paginate(**input)` returns a `PageIterator`; each `next()` sends one request and injects the previous output token as the next input token.
- SDK v3's `paginate<Operation>` is a generated async generator that does the same token threading, yielding each page as it arrives.
- Both stop when the service stops returning a next token (`IsTruncated: false` or a null `NextToken`).
### PaginationConfig and Its v3 Equivalents
| boto3 `PaginationConfig` | SDK v3 config | Meaning |
|--------------------------|---------------|---------|
| `PageSize` | `pageSize` | Items requested per page |
| `MaxItems` | (manual break) | Overall cap across pages |
| `StartingToken` | `startingToken` | Resume from a saved cursor |
### Flattening Items with search (boto3)
boto3 paginators expose `.search(jmespath)`, which runs a JMESPath expression across every page and yields matching values as a flat stream.
<CodeGroup labels="Python (boto3), TypeScript (AWS SDK v3)">
```python
# --- Python (boto3) ---
import boto3
s3 = boto3.client("s3")
paginator = s3.get_paginator("list_objects_v2")
# Yield only keys larger than 1 MiB, flattened across all pages.
big = paginator.paginate(Bucket="my-bucket").search("Contents[?Size > `1048576`].Key")
for key in big:
print(key)
```
```typescript
// --- TypeScript (AWS SDK v3) ---
import { S3Client, paginateListObjectsV2 } from "@aws-sdk/client-s3";
const s3 = new S3Client({});
// No .search in v3: filter and collect while iterating.
const big: string[] = [];
for await (const page of paginateListObjectsV2({ client: s3 }, { Bucket: "my-bucket" })) {
for (const obj of page.Contents ?? []) {
if (obj.Key && (obj.Size ?? 0) > 1_048_576) big.push(obj.Key);
}
}
console.log(big);
```
</CodeGroup>
## Gotchas
- **Reading only the first response** - a bare `list_objects_v2` returns one page. **Fix:** use the paginator or check `IsTruncated` in a manual loop.
- **Wrong token field in manual loops** - S3 uses `ContinuationToken` / `NextContinuationToken`, while many services use `NextToken`. **Fix:** prefer the paginator, which knows the right fields.
- **Mutating shared input across pages** - reusing and mutating one input dict can corrupt token handling. **Fix:** let the paginator own the input; do not inject tokens yourself.
- **Buffering everything** - collecting a huge listing into one list defeats pagination. **Fix:** process items per page or stream them out.
- **Assuming SDK v3 has .search** - it does not. **Fix:** filter with normal code while iterating pages.
- **Expecting MaxItems in v3** - there is no overall cap argument. **Fix:** count and `break` when you reach your limit.
## Alternatives
| Alternative | Use When | Don't Use When |
|-------------|----------|----------------|
| Automatic paginator | You want all pages with minimal code | You must persist a cursor between separate runs |
| Manual `NextToken` loop | You need to save and resume a cursor across invocations | A paginator already covers the case (most of the time) |
| `.search` JMESPath (boto3) | You want a flat, filtered item stream in Python | You are in SDK v3, or need complex per-item logic |
| Single call, no pagination | The operation is guaranteed to return one bounded page | The operation can return a continuation token |
## FAQs
<details>
<summary>How do I know if an operation is paginated?</summary>
If its response includes a token field (`NextToken`, `NextContinuationToken`, `Marker`) or an `IsTruncated` flag, it paginates. boto3 also errors clearly if you call `get_paginator` on an operation that has no paginator.
</details>
<details>
<summary>What does each iteration of the paginator yield?</summary>
A full page response object, the same shape a single call would return. You then loop the result key (`Contents`, `Items`, `Reservations`, and so on) inside it.
</details>
<details>
<summary>Does the paginator handle retries?</summary>
Each page is a normal request, so it uses the client's retry configuration. Throttling on any page is retried with backoff just like a standalone call.
</details>
<details>
<summary>How do I resume pagination later?</summary>
Save the last output token and pass it as `StartingToken` (boto3) or `startingToken` (SDK v3) on the next run. This is the main reason to keep a manual cursor.
</details>
<details>
<summary>Why does SDK v3 have a helper per operation?</summary>
v3 is code-generated and modular, so each paginatable operation gets its own strongly typed `paginate<Operation>` function exported from the client package.
</details>
<details>
<summary>Can I set page size to reduce cost?</summary>
Page size changes items per request, not total items, so it mainly affects memory and request count. To reduce data scanned, filter server-side with `Prefix`, `KeyConditionExpression`, or a `FilterExpression`.
</details>
<details>
<summary>Is .search available in SDK v3?</summary>
No. JMESPath `.search` is a boto3 paginator feature. In v3 you filter items with ordinary TypeScript while iterating the pages.
</details>
<details>
<summary>What happens if I forget IsTruncated in a manual loop?</summary>
You process only the first page and silently miss the rest. This is the classic pagination bug, and it is exactly what the paginator prevents.
</details>
## Related
- [Pagination & Retries Basics](./pagination-and-retries-basics.md) - the first paginated call, step by step.
- [Waiters: Polling for Resource State Changes](./waiters-polling-for-resource-state-changes.md) - the other built-in loop the SDKs provide.
- [Structured Error Handling: ClientError vs SDK v3 Exceptions](./structured-error-handling-clienterror-vs-sdk-v3-exceptions.md) - handling per-page errors.
- [Pagination, Waiters & Retries Best Practices](./best-practices.md) - defaults to set explicitly.
> **Stack versions:** This page was written for **boto3 1.43.x** (Python 3.10+) and the **AWS SDK for JavaScript v3** (Node.js 18+).
</content>