That single fact reshapes how you call it. A remote, multi-tenant API cannot hand you an unbounded list in one response, cannot finish a resource change the instant it replies, and cannot let one caller consume unlimited capacity. Paginators, waiters, and retry logic are the three tools that both SDKs provide to work with those realities instead of fighting them.
## Summary
- AWS service APIs are **list-heavy, rate-limited, and eventually consistent**, which breaks the naive assumption that one call returns one complete answer.
- **Insight:** paginators, waiters, and retries are not optional niceties - they are the SDK's answer to three concrete API behaviors you will hit in production.
- **Key Concepts:** **pagination** (bounded pages + continuation tokens), **waiters** (poll until a target state), **retries** (backoff on throttling and transient errors), **eventual consistency**, **throttling**.
- **When to Use:** any list operation, any operation that changes resource state, and any call that can be throttled - which in practice is nearly all of them.
- **Limitations/Trade-offs:** these tools add latency and hidden control flow; used carelessly they mask real failures or run up cost through tight polling.
- **Related Topics:** structured error handling, exponential backoff configuration, and idempotency tokens for safe retries.
## Foundations
Three properties of AWS APIs drive everything on this page.
**They return bounded pages.** A `ListObjectsV2`, `DescribeInstances`, or `Scan` call does not return every item. It returns up to a service-defined maximum (often 1,000) plus a token that means "there is more." If you read only the first response, you silently process a fraction of the data. Pagination is how you follow that token to completion.
**They settle asynchronously.** When you call `RunInstances` or `CreateTable`, the API responds quickly with a resource that is `pending` or `CREATING`, not `running` or `ACTIVE`. The work continues on AWS's side after the response. If your next line of code assumes the resource is ready, it fails. Waiters exist to bridge that gap by polling until the resource reaches the state you need.
**They throttle and drop.** AWS meters request rates per account, per service, and per operation. Exceed the limit and you get a `ThrottlingException` or `429`. Add ordinary internet flakiness - reset connections, timeouts, transient `500`s - and a fraction of calls will fail for reasons that have nothing to do with your code. Retry logic with backoff turns those expected failures into successful calls a moment later.
None of these are edge cases. They are the normal shape of a large, multi-tenant, distributed system, and both boto3 and AWS SDK v3 build the tooling in.
## Mechanics & Interactions
Here is what "the whole result set" actually requires. The manual version below shows the loop the paginator replaces, so you can see what the tool is doing for you.
<CodeGroup labels="Python (boto3), TypeScript (AWS SDK v3)">
```python
# --- Python (boto3) ---
import boto3
s3 = boto3.client("s3")
token = None
while True:
kwargs = {"Bucket": "my-bucket"}
if token:
kwargs["ContinuationToken"] = token
resp = s3.list_objects_v2(**kwargs)
for obj in resp.get("Contents", []):
print(obj["Key"])
if not resp.get("IsTruncated"):
break
token = resp["NextContinuationToken"]
```
```typescript
// --- TypeScript (AWS SDK v3) ---
import { S3Client, ListObjectsV2Command } from "@aws-sdk/client-s3";
const s3 = new S3Client({});
let token: string | undefined = undefined;
do {
const resp = await s3.send(
new ListObjectsV2Command({ Bucket: "my-bucket", ContinuationToken: token }),
);
for (const obj of resp.Contents ?? []) console.log(obj.Key);
token = resp.IsTruncated ? resp.NextContinuationToken : undefined;
} while (token);
```
</CodeGroup>
That token-passing loop is easy to get subtly wrong: forgetting `IsTruncated`, using the wrong token field name, or mutating shared input. Paginators encapsulate exactly this, and you will see the clean version in the next articles.
The three tools also interact. A paginator issues many requests, so each page is itself subject to throttling and therefore rides on the same retry machinery. A waiter is a controlled polling loop whose every poll is a real API call that can be throttled and retried. And retries only stay safe when the operation being retried is idempotent - which is why idempotency tokens sit alongside these tools rather than apart from them.
## Advanced Considerations & Applications
At scale, the reason these tools matter shifts from correctness to cost and stability.
- **Pagination controls blast radius.** A `Scan` over a large table, paginated with server-side filtering and a sane page size, reads far less than pulling everything and filtering in memory. The paginator is where you set `PaginationConfig` / `pageSize` to cap consumed capacity.
- **Waiters replace hand-written polling.** A naive `while not ready: sleep(1)` loop hammers the API and burns request cost. A waiter uses a bounded delay, a maximum attempt count, and known success and failure states, so it stops promptly and predictably.
- **Retries need backoff and jitter.** Immediate, synchronized retries from many clients create a thundering herd that keeps the service throttled. Exponential backoff with jitter spreads them out. Both SDKs do this by default in `standard` mode and add rate estimation in `adaptive` mode.
- **Eventual consistency is not just create/delete.** A newly created IAM role or S3 object may not be immediately visible to a follow-up read. Waiters and short, bounded retries absorb that window instead of failing the whole workflow.
The practical takeaway: you almost never disable these tools. You configure them - page size, max attempts, retry mode, waiter timeout - to match the workload.
## Common Misconceptions
- **"One list call returns everything."** It returns one page plus a token. Ignoring the token means silently processing partial data.
- **"If the create call succeeded, the resource is ready."** The call succeeded; the resource is usually still transitioning. Wait for the target state before using it.
- **"Retries mean my code is broken."** Throttling and transient errors are expected at scale. Built-in retries are a design feature, not a symptom.
- **"I should write my own retry loop."** Both SDKs already retry with backoff and jitter. Reinventing it usually removes the jitter and makes things worse.
- **"Polling in a tight loop is fine."** Tight loops multiply request cost and can keep you throttled. Use waiters or backed-off polling.
## FAQs
<details>
<summary>Why can't AWS just return the whole list?</summary>
Bounded pages keep responses small and predictable for a shared, multi-tenant service. The continuation token lets you retrieve the rest at a pace the service can serve, without one caller monopolizing capacity.
</details>
<details>
<summary>What is eventual consistency in this context?</summary>
After a write or a create, there is a brief window where reads may not yet reflect the change. Waiters and short bounded retries absorb that window so your workflow does not fail on a race.
</details>
<details>
<summary>Are paginators, waiters, and retries specific to Python or JavaScript?</summary>
No. They are concepts baked into every AWS SDK. boto3 and AWS SDK v3 differ only in surface syntax; the underlying API behavior they address is identical.
</details>
<details>
<summary>Do I have to configure all three for every call?</summary>
No. Use a paginator for list operations, a waiter for state changes, and rely on the built-in retry defaults everywhere. You tune them only when the workload demands it.
</details>
<details>
<summary>Is throttling the same as an error in my code?</summary>
No. A `ThrottlingException` means you exceeded a rate limit, which is expected under load. The SDK retries it with backoff, and it usually succeeds on a later attempt.
</details>
<details>
<summary>Why does backoff need jitter?</summary>
Without jitter, many clients retry at the same instants and re-throttle the service in synchronized waves. Random jitter spreads retries out so the service can recover.
</details>
<details>
<summary>When are retries unsafe?</summary>
When the operation is not idempotent, a blind retry can duplicate an effect (like creating two resources). Idempotency tokens make such operations safe to retry.
</details>
<details>
<summary>Where do I learn the concrete syntax?</summary>
The next articles cover paginators, waiters, backoff configuration, and idempotency tokens with runnable examples in both SDKs. This page is the why behind them.
</details>
## Related
- [Pagination & Retries Basics](./pagination-and-retries-basics.md) - your first paginated call and throttling handler.
- [Paginators in boto3 & SDK v3](./paginators-in-boto3-and-sdk-v3.md) - automatic iteration versus manual tokens.
- [Waiters: Polling for Resource State Changes](./waiters-polling-for-resource-state-changes.md) - polling done right.
- [Exponential Backoff & Jitter Configuration](./exponential-backoff-and-jitter-configuration.md) - tuning the retry strategies.
> **Stack versions:** This page was written for **boto3 1.43.x** (Python 3.10+) and the **AWS SDK for JavaScript v3** (Node.js 18+).
</content>
<span class="text-xs italic text-foreground/50" aria-hidden="true">Y29kZWd1aWRlcy5pb3xjZ2lvMTEwNnwyMDI2MDc</span>