Most data cools off. A log written today is read constantly this week, occasionally next month, and almost never after a year - but if it sits in S3 Standard the whole time, you pay the top rate for storage nobody touches.
Lifecycle rules fix this without any code running on your side. You declare, once, that objects transition to cheaper storage classes as they age and expire when they are no longer needed. S3 enforces the policy continuously. This page shows how to write those rules with the SDK, what the storage classes cost you in trade-offs, and the gotchas that surprise people.
A lifecycle configuration is an array of rules. Each rule has a Filter (prefix, tag, or both via And), a Status, and one or more actions.
Transitions move an object to a cheaper storage class after N days. Expiration deletes it. Both are keyed off the object's age since creation.
The whole configuration is replaced on every Put - there is no partial update, so always send the full desired rule set.
S3 evaluates rules asynchronously, typically once a day. An object crosses its transition or expiration boundary within about a day of the threshold, not to the exact second.
The cheaper the class, the longer the minimum storage duration and the more retrieval costs. Transitioning too aggressively can cost more than it saves if objects are deleted or read before the minimum period.
On a versioned bucket, Expiration on a current object just adds a delete marker - it does not free space. To actually reclaim storage, use NoncurrentVersionTransition and NoncurrentVersionExpiration to tier and remove old versions, and ExpiredObjectDeleteMarker to clean up dangling markers.
Put replaces, it does not append - a second PutBucketLifecycleConfiguration wipes the previous rules. Fix: read the current config, modify it, and put the complete set back.
Expiration on a versioned bucket frees nothing - it only adds a delete marker over the current version. Fix: add NoncurrentVersionExpiration (and expire delete markers) to reclaim space.
Transition minimums cost money - moving to STANDARD_IA then deleting at day 20 still bills the 30-day minimum. Fix: transition only objects that will outlive the class's minimum duration.
Small objects and IA - STANDARD_IA bills a 128 KB minimum per object, so tiering many tiny files can raise costs. Fix: keep small, short-lived objects in Standard, or use Intelligent-Tiering.
Transitions are not instant - rules run about once a day, so do not expect a change exactly at the day boundary. Fix: treat thresholds as "within a day," and verify with GetBucketLifecycleConfiguration, not by watching a single object.
Filter shape confusion - combining prefix and tags requires an And wrapper. Fix: use Filter.And with Prefix and Tags when you need multiple conditions.
The evaluation is free; you pay only the per-object transition request fee and the destination class's storage and retrieval costs. For cold data this is almost always a net saving.
Why did my Put wipe my other lifecycle rules?
PutBucketLifecycleConfiguration replaces the entire configuration. It is not additive. Read the current rules, add or edit, and send the complete set back every time.
How quickly does a transition happen?
Asynchronously, usually within a day of the object reaching the threshold age. S3 processes lifecycle actions in a daily batch, so it is never to-the-second.
Which storage class should I transition to first?
STANDARD_IA after about 30 days is the common first step for data that is still occasionally read. Move to GLACIER_IR, GLACIER, or DEEP_ARCHIVE as access drops and retrieval latency becomes acceptable.
Why did tiering small files increase my bill?
STANDARD_IA and Glacier classes bill a minimum object size (128 KB for IA) and minimum storage durations. Many tiny or short-lived objects can cost more tiered than left in Standard. Use Intelligent-Tiering or leave them in Standard.
Does expiration delete objects on a versioned bucket?
Not really. It adds a delete marker over the current version, hiding it but keeping the data. Use NoncurrentVersionExpiration and expire delete markers to actually free storage.
Can I target objects by tag instead of prefix?
Yes. A Filter can match a tag, a prefix, or both. To combine conditions, wrap them in Filter.And with Prefix and Tags.
How do I clean up failed multipart uploads?
Add AbortIncompleteMultipartUpload with DaysAfterInitiation to a rule. S3 will abort and delete orphaned parts that would otherwise bill invisibly.
What is Intelligent-Tiering and when should I use it?
It is a storage class that moves objects between access tiers automatically based on observed usage, for a small monitoring fee. Use it when access patterns are unknown; use explicit lifecycle rules when they are predictable.
How do I remove a lifecycle configuration?
Call DeleteBucketLifecycle (boto3 delete_bucket_lifecycle, SDK v3 DeleteBucketLifecycleCommand) to clear all rules on the bucket.