Reading DynamoDB well comes down to one distinction: Query targets a single partition and stays cheap at any scale, while Scan reads the whole table. Filters confuse people because they look like a WHERE clause but bill nothing like one.
This page shows efficient queries, correct pagination, where filters really apply, and the rare cases where a Scan is fine - with runnable code in both SDKs using the document interface.
Query one customer's orders newest-first, paginate through a large result set with LastEvaluatedKey, then contrast with a Scan that reads the whole table.
# --- Python (boto3) ---import boto3from boto3.dynamodb.conditions import Key, Attrtable = boto3.resource("dynamodb", region_name="us-east-1").Table("AppData")# 1. Query with a key condition + sort-key range + reverse order.resp = table.query( KeyConditionExpression=Key("pk").eq("CUSTOMER#c-1") & Key("sk").begins_with("ORDER#"), ScanIndexForward=False, # descending: newest first Limit=50,)print([i["sk"] for i in resp["Items"]])# 2. Paginate: keep passing LastEvaluatedKey until it is absent.items, start_key = [], Nonewhile True: kwargs = {"KeyConditionExpression": Key("pk").eq("CUSTOMER#c-1")} if start_key: kwargs["ExclusiveStartKey"] = start_key page = table.query(**kwargs) items.extend(page["Items"]) start_key = page.get("LastEvaluatedKey") if not start_key: breakprint("total items:", len(items))# 3. Scan reads the WHOLE table; the filter runs after the read.scanned = table.scan(FilterExpression=Attr("total").gt(100))print("matched (but paid to read all):", len(scanned["Items"]))
This is the single most important idea on the page.
A KeyConditionExpression operates on the partition key (equality only) and the sort key (equality, comparison, begins_with, between). DynamoDB uses it to find items, so you are metered only on the items it matches. It is where efficiency lives.
A FilterExpression runs on non-key attributes after DynamoDB has already read and metered the items. It reduces the payload returned to you but never the read cost. Filtering a million items down to ten still bills for a million items' worth of reads. Treat filters as a convenience for trimming an already-cheap result, never as a way to make an expensive read cheap.
DynamoDB returns at most 1 MB of data per read call. If more matches exist, the response includes a LastEvaluatedKey. Pass it back as ExclusiveStartKey to get the next page, and stop when it is absent. A Limit caps items per page but does not change this loop. Note that Limit applies before filtering, so a filtered page can come back with fewer items than Limit yet still have a LastEvaluatedKey.
ProjectionExpression asks DynamoDB to return only named attributes, cutting network bytes (though not read-capacity units, which are based on item size read). Use #name placeholders for reserved words. Add ConsistentRead=True to a base-table or LSI Query when you must see the latest write; GSIs are always eventually consistent.
When a Scan is genuinely required (a migration, an export, a small table), you can parallelize it by splitting the table into segments and scanning each concurrently.
# --- Python (boto3) ---# Worker 0 of 4: each worker scans a disjoint segment concurrently.page = table.scan(Segment=0, TotalSegments=4)print(len(page["Items"]))
// --- TypeScript (AWS SDK v3) ---import { ScanCommand } from "@aws-sdk/lib-dynamodb";// Worker 0 of 4: each worker scans a disjoint segment concurrently.const page = await doc.send(new ScanCommand({ TableName: "AppData", Segment: 0, TotalSegments: 4 }));console.log((page.Items ?? []).length);
Expecting a filter to save money. A FilterExpression runs after the read and bills for every item scanned. Fix: move the condition into a KeyConditionExpression or a GSI.
Reading only the first page. DynamoDB caps a response at 1 MB. Fix: loop on LastEvaluatedKey/ExclusiveStartKey until it is absent.
Limit surprises with filters.Limit counts items examined before filtering, so a filtered page may return fewer rows than Limit. Fix: keep paginating while LastEvaluatedKey is present.
Scan on a hot path. Scans read the entire table and slow down as it grows. Fix: design a key or index so the read is a Query.
Reserved words in expressions.status, name, size, ttl, and others are reserved. Fix: use ExpressionAttributeNames placeholders.
Assuming a GSI query is strongly consistent. GSIs are eventually consistent. Fix: query the base table or an LSI with ConsistentRead=True when freshness matters.
What is the core difference between Query and Scan?
Query reads only items under one partition key you supply and is efficient at any scale. Scan reads every item in the table (or index) and its cost grows with the data, regardless of how many items you keep.
Does a FilterExpression reduce read cost?
No. Filters are applied after DynamoDB reads and meters the items, so they reduce the payload returned but not the capacity consumed. To reduce cost, narrow with a KeyConditionExpression or an index.
How does pagination work?
Each read returns up to 1 MB. If more results exist, the response includes a LastEvaluatedKey. Pass it back as ExclusiveStartKey on the next call and repeat until the response omits it.
Why did my page return fewer items than Limit?
Limit counts items examined before the filter runs. If a FilterExpression removes some, the page returns fewer than Limit even though more matching items may exist - keep paginating on LastEvaluatedKey.
How do I get results newest-first?
Set ScanIndexForward to false on a Query. Items are stored ascending by sort key, so this returns them in descending order without any client-side sorting.
What does ProjectionExpression do?
It limits the attributes returned to only those you name, reducing network payload. It does not reduce read-capacity units, which are based on the size of the item read, not the size returned.
When is a Scan acceptable?
For small tables, one-off maintenance, migrations, or full exports where reading everything is the intent. For large or hot tables, use a parallel Scan if you must, but prefer designing a Query-able key or index.
Can I query a Global Secondary Index the same way?
Yes. Pass IndexName and a KeyConditionExpression on the index's keys. The only difference is that GSIs are always eventually consistent, so you cannot request a strongly consistent read on them.