S3 Beyond Put/Get: The Operations That Matter at Scale
PutObject and GetObject are the first two S3 calls anyone learns, and for a while they are all you need.
//
Busca en todas las páginas de la documentación
PutObject and GetObject are the first two S3 calls anyone learns, and for a while they are all you need.
Then reality arrives. Files grow past a few gigabytes. A browser needs to upload directly without routing bytes through your servers. Storage bills climb because nothing is ever tiered or expired. A new object should kick off a pipeline instead of sitting until something polls for it. And one day you need to change the ACL on ten million objects.
None of those problems is solved well by put and get alone. This page maps the wider S3 surface - the operations that separate a toy bucket from a production storage layer - and explains how they relate before the following pages take each one in depth.
boto3 transfer manager and the SDK v3 Upload helper do this for you.GET or PUT one object directly, with no AWS credentials and no proxy in the middle.Every advanced S3 feature answers a question that put and get cannot.
How do I move an object too big to send in one request? That is multipart upload. A single PutObject is capped at 5 GB and, more practically, sending 4 GB as one stream means one network blip restarts everything. Multipart breaks the object into parts (5 MB minimum each, up to 10,000 parts), uploads them in parallel, and assembles them server-side. The transfer managers wrap this so you call one method and never see the parts.
How do I let a client I do not trust read or write an object? That is a presigned URL. Your backend, which does hold credentials, signs a URL that grants access to exactly one object and one operation for a set number of seconds. The browser uses that URL directly against S3. Your servers never see the file bytes and never ship a credential to the client.
How do I stop paying full price for data nobody reads? That is a lifecycle rule. You declare, once, that objects under a prefix move to Infrequent Access after 30 days, to Glacier after 90, and expire after a year. S3 enforces it continuously with no code running on your side.
How do I react the moment an object lands? That is event notifications. A bucket can publish s3:ObjectCreated:* and s3:ObjectRemoved:* events to Lambda, SQS, or SNS, or route every event to EventBridge for richer filtering and fan-out. The upload becomes the trigger.
How do I query inside an object, or act on all of them at once? Those are the two extremes. S3 Select reads rows and columns out of a single CSV, JSON, or Parquet object without downloading the whole thing - though it is now a legacy API. S3 Batch Operations runs one action (copy, tag, restore, invoke a Lambda) across a manifest of millions of objects as a managed, retryable job.
These features are not a menu of unrelated tricks. They compose into recognizable production patterns.
Consider a user-generated content pipeline. A browser asks your API for a presigned PUT URL and uploads a large video directly to S3 - and because the SDK's upload helper uses multipart under the hood, that large file arrives reliably in parallel parts. The moment the object is complete, an event notification fires to a Lambda function, which transcodes the video and writes derivatives back. A lifecycle rule transitions the original to Glacier after 30 days because it is rarely watched again, and expires temporary render artifacts after a week.
That single flow uses four of the five advanced areas, and each does the job the core API could not. Presigning keeps bytes off your servers. Multipart makes the large upload survivable. Events remove polling. Lifecycle removes the cost cleanup you would otherwise script.
The shared thread is that S3 is doing more of the work. Put and get treat S3 as a dumb blob store your code drives. The advanced surface treats S3 as an active participant - it retries parts, enforces policies, emits events, and runs jobs - which is exactly what you want as volume grows and manual operation stops scaling.
The SDK patterns still transfer. Every one of these is the same client-request-response shape you already know, with the same credential resolution and the same paginators for large listings. Multipart and presigning add small dedicated helpers (@aws-sdk/lib-storage and @aws-sdk/s3-request-presigner in v3; TransferConfig and generate_presigned_* in boto3), and Batch Operations uses a separate control-plane client (@aws-sdk/client-s3-control), but the mental model does not change.
A useful way to hold the surface in your head is by the problem each solves.
| Problem | Feature | What S3 does for you |
|---|---|---|
| Objects too big for one request | Multipart / transfer manager | Splits, parallelizes, and retries parts |
| Untrusted client needs direct access | Presigned URLs | Grants scoped, time-limited access with no credentials |
| Paying full price for cold data | Lifecycle rules | Tiers and expires objects automatically |
| Reacting to new objects | Event notifications / EventBridge | Triggers downstream compute on create/delete |
| Reading inside one object | S3 Select (legacy) | Returns rows/columns without a full download |
| Acting on millions of objects | S3 Batch Operations | Runs one operation across a manifest as a job |
Two of these deserve an early caveat. S3 Select still works, but AWS now steers new work toward Amazon Athena, S3 Object Lambda, or simply filtering client-side, so treat it as a legacy option rather than a default. And presigned URLs are powerful precisely because they bypass your usual authorization - a leaked URL is valid access until it expires, so short lifetimes and tight scoping matter.
The rest are close to free wins. Multipart is often automatic the moment you use the transfer helper. Lifecycle rules are pure configuration with no runtime cost. Event notifications replace polling loops that would otherwise cost both money and latency.
Beyond this section, the same instincts extend further - S3 Transfer Acceleration for cross-region uploads, Object Lambda for transforming data on read, Storage Lens for fleet-wide analytics. But they build on the same idea this section is really about: pushing work into S3 instead of pulling everything into your own code.
When an object is large enough that a single request is unreliable, when a client other than your server must access the bytes, when storage cost needs automatic control, or when a new object should trigger downstream work. Each is a different advanced feature.
Usually no. The boto3 transfer manager (upload_file with TransferConfig) and the SDK v3 Upload helper from @aws-sdk/lib-storage handle splitting, parallelism, and retries. The low-level part APIs exist for advanced control.
They are safe when scoped and short-lived, but anyone holding a valid URL has the access it grants until it expires. Use the shortest practical expiry and generate them per operation and per object.
It still functions, but AWS now recommends Athena, S3 Object Lambda, or client-side filtering for querying object data. New designs should prefer those; S3 Select is best left to existing code that already uses it.
Use direct notifications (Lambda/SQS/SNS) for simple point-to-point triggers. Use EventBridge when you want richer filtering, multiple targets, replay, or to route S3 events alongside the rest of your event bus.
They cost nothing to run and cut storage bills continuously by tiering cold data to cheaper classes and expiring what you no longer need. Setting them at bucket creation avoids a large cleanup project later.
Up to 5 TB, assembled from as many as 10,000 parts. A single PutObject without multipart is capped at 5 GB, which is the practical reason to switch to the transfer helper for large files.
No. Lifecycle rules run continuously against age-based conditions. Batch Operations runs one explicit action (copy, tag, restore, invoke Lambda) over a fixed manifest of objects as a one-time managed job.
Not fundamentally. They are the same client-request-response calls, with a few dedicated helpers (lib-storage, s3-request-presigner) and one separate control-plane client for Batch Operations. Credentials and paginators behave exactly as elsewhere.
With whichever problem you actually have. Large uploads point to multipart; direct client access points to presigning; cost points to lifecycle; automation points to events. The Basics page next shows the bulk list and delete operations these build on.
Stack versions: This page was written for boto3 1.43.x (Python 3.10+) and the AWS SDK for JavaScript v3 (Node.js 18+).
Revisado por Chris St. John·Última actualización: 23 jul 2026