Busca en todas las páginas de la documentación
Every accelerator AWS offers for AI workloads - Nvidia GPUs, AWS Trainium, and AWS Inferentia - shows up in your SDK code the same way: as an EC2 instance type string passed to RunInstances. There is no separate "AI service" API to learn for provisioning the hardware itself.
The decision that actually matters is not which API to call. It is which instance family fits your workload's framework, cost profile, and capacity needs. This page lays out that decision from first principles.
RunInstances with an instance type string, exactly like any other EC2 instance.Three accelerator families cover almost all AI workloads on AWS, and each maps to a family of EC2 instance types.
| Family | Instance types | Built by | Primary use |
|---|---|---|---|
| GPU | p5, p6 (large-scale), g5, g6 (smaller-scale) | Nvidia | Training and inference, any CUDA framework |
| Trainium | trn1, trn2 | AWS | Large-scale training, cost-optimized |
| Inferentia | inf1, inf2 | AWS | High-throughput, low-cost inference |
All three are launched with the same operation. Only the InstanceType string changes.
# --- Python (boto3) ---
import boto3
ec2 = boto3.client("ec2", region_name="us-east-1")
# Same call, three different accelerator families - only InstanceType changes.
for instance_type in ["p5.48xlarge", "trn1.32xlarge", "inf2.48xlarge"]:
print(instance_type)// --- TypeScript (AWS SDK v3) ---
import { EC2Client } from "@aws-sdk/client-ec2";
const ec2 = new EC2Client({ region: "us-east-1" });
// Same call, three different accelerator families - only InstanceType changes.
for (const instanceType of ["p5.48xlarge", "trn1.32xlarge", "inf2.48xlarge"]) {
console.log(instanceType);
}The instance type is where the similarity ends. What runs on top of it differs sharply between GPU and the AWS-built silicon.
GPU instances (p5, p6, g5, g6) run Nvidia hardware, so any framework built on CUDA - PyTorch, TensorFlow, JAX - runs with little or no code change. This is why GPUs remain the default choice: the software ecosystem assumes CUDA, and a GPU instance is the path of least resistance.
Trainium (trn1, trn2) and Inferentia (inf1, inf2) are AWS's own accelerator silicon, purpose-built to cut cost-per-token for training and inference respectively. They do not run CUDA. Your model must be compiled through the AWS Neuron SDK, which provides framework plugins for PyTorch and TensorFlow that translate model operations onto the Neuron architecture.
That compilation step is the real trade-off. A model that compiles cleanly through Neuron can see meaningfully lower cost-per-token than an equivalent GPU instance. A model that leans on custom CUDA kernels, exotic ops, or a framework Neuron does not support may not compile at all, or may need rework before it does.
Provisioning itself never changes based on this choice - it is still RunInstances, DescribeInstances, and the same waiters covered in the EC2 section. What changes is everything above the EC2 API: the container image, the framework build, and the Neuron compilation step, none of which are AWS SDK calls.
Training vs inference is the first fork. Trainium (trn1/trn2) targets training throughput; Inferentia (inf1/inf2) targets inference throughput and latency. Using Trainium to serve inference or Inferentia to train a large model from scratch is possible in some cases but works against the hardware's design point - GPUs are usually the safer general-purpose choice when a workload does not cleanly fall into one bucket.
Scale matters. p5/p6 and trn1/trn2 target large, multi-node training runs with high-bandwidth networking between instances. g5/g6 cover smaller-scale GPU training and inference, and inf1/inf2 cover inference at a range of throughput levels. Picking the largest instance type available is rarely the right default; match the instance to the actual model size and batch throughput you need.
Capacity is often the real constraint. p5/p6 and Trainium capacity at large scale is frequently the scarcest resource, not the cheapest. A team that has validated Neuron compatibility but cannot get on-demand trn1 capacity when needed may still end up choosing GPUs, or reserving capacity ahead of time. See Capacity Reservations & Capacity Blocks for ML for how to lock in a instance type and AZ (or a future time window) ahead of a training run.
Engineering cost is the hidden line item. The headline price-per-hour difference between a GPU instance and an equivalent Trainium instance is real, but it is not the whole story. Validating a Neuron compilation, debugging unsupported ops, and maintaining a second build pipeline alongside your CUDA one all cost engineering time. For a one-off experiment, that cost may not be worth it. For a workload running continuously at scale, it usually is.
A practical decision path: start on GPU to get a model working end to end, measure your actual cost-per-token at target scale, and only then evaluate whether a Neuron port of that specific workload is worth the engineering investment.
RunInstances call as any other instance.No. All three are provisioned as EC2 instance types via RunInstances, DescribeInstances, and the standard EC2 waiters. Only the InstanceType value differs.
The software layer that compiles PyTorch or TensorFlow models to run on Trainium and Inferentia hardware. It is a framework plugin, not an EC2 or Bedrock API, and sits entirely on the software side of your stack.
When your framework or model relies on CUDA-only features, when you need results fast without a compilation step, or when Neuron compatibility for your specific architecture is unproven.
For large-scale training workloads where the model compiles cleanly through Neuron and the cost-per-token savings at your training volume justify the engineering investment.
For high-throughput inference serving where a validated Neuron-compiled model can significantly lower cost-per-token compared to an equivalent GPU instance.
Yes, especially for p5/p6 and large Trainium clusters. See Capacity Reservations & Capacity Blocks for ML for reserving scarce capacity ahead of a training run.
Yes. They are independent EC2 instance families with no special interaction; you can run GPU training and Inferentia inference side by side.
Stack versions: This page was written for boto3 1.43.x (Python 3.10+) and the AWS SDK for JavaScript v3 (Node.js 18+).
Revisado por Chris St. John·Última actualización: 25 jul 2026