Batch-Loading Embeddings at Scale

Q: What throttles a large load?

Usually Bedrock embedding quota first, then the put_vectors request rate. Add backoff-and-retry on both, and size embedding concurrency to your model's limit to avoid 429s.

Loading one vector at a time is fine for a demo and ruinous for a corpus. A million documents means a million embeddings and a million writes, and doing either serially, one item per request, turns a minutes-long job into an hours-long one.

This page builds a practical ingestion pipeline: read source items, embed them in parallel with Bedrock, group them into batched put_vectors calls, and make the whole thing resumable and idempotent. The embedding calls use the stable Bedrock API; the S3 Vectors write is a 2026 surface, so batch limits carry a verify-at-build-time note.

Recipe

Quick-reference recipe card - batch many vectors into one write.

# --- Python (boto3) ---
import boto3
 
sv = boto3.client("s3vectors")
 
# Group vectors and write them in one call instead of one-per-request.
batch = [
    {"key": doc_id, "data": {"float32": vec}, "metadata": meta}
    for doc_id, vec, meta in rows
]
sv.put_vectors(vectorBucketName="my-vectors", indexName="docs", vectors=batch)
# Confirm the max vectors per put_vectors call against current boto3 docs (2026 API surface).

// --- TypeScript (AWS SDK v3) ---
import { S3VectorsClient, PutVectorsCommand } from "@aws-sdk/client-s3vectors";
 
const sv = new S3VectorsClient({});
 
const batch = rows.map(({ docId, vec, meta }) => ({
  key: docId, data: { float32: vec }, metadata: meta,
}));
await sv.send(new PutVectorsCommand({ vectorBucketName: "my-vectors", indexName: "docs", vectors: batch }));
// Confirm the max vectors per put_vectors call against current @aws-sdk/client-s3vectors docs (2026 API surface).

When to reach for this:

Initial load of an existing document corpus into a new index.
Nightly or streaming top-ups as new documents arrive.
Re-embedding after switching models (a new index plus a full reload).
Backfilling metadata or re-keying an existing dataset.

Working Example

An end-to-end batch loader: chunked embedding with bounded concurrency, batched writes, retries, and a stable key so re-runs upsert. Shown in Python with a smaller illustrative TypeScript equivalent.

# --- Python (boto3) ---
import boto3, json, time
from concurrent.futures import ThreadPoolExecutor
 
bedrock = boto3.client("bedrock-runtime")
sv = boto3.client("s3vectors")
 
BATCH = 200   # vectors per put_vectors call - verify the service max at build time (2026 API)
 
def embed(text: str) -> list[float]:
    resp = bedrock.invoke_model(
        modelId="amazon.titan-embed-text-v2:0",
        body=json.dumps({"inputText": text}),
    )
    return json.loads(resp["body"].read())["embedding"]
 
def put_with_retry(vectors, attempts=5):
    for i in range(attempts):
        try:
            sv.put_vectors(vectorBucketName="my-vectors", indexName="docs", vectors=vectors)
            return
        except Exception:
            if i == attempts - 1:
                raise
            time.sleep(2 ** i)   # exponential backoff
 
def load(docs):   # docs: list[{"id":..., "text":..., "topic":...}]
    with ThreadPoolExecutor(max_workers=8) as pool:
        embeddings = list(pool.map(lambda d: embed(d["text"]), docs))
 
    batch = []
    for d, vec in zip(docs, embeddings):
        batch.append({
            "key": d["id"],                                   # stable key -> upsert on re-run
            "data": {"float32": vec},
            "metadata": {"topic": d["topic"], "raw_text": d["text"]},
        })
        if len(batch) >= BATCH:
            put_with_retry(batch)
            batch = []
    if batch:
        put_with_retry(batch)
 
# load(read_corpus())  # feed your source iterator

// --- TypeScript (AWS SDK v3) ---
import { BedrockRuntimeClient, InvokeModelCommand } from "@aws-sdk/client-bedrock-runtime";
import { S3VectorsClient, PutVectorsCommand } from "@aws-sdk/client-s3vectors";
 
const bedrock = new BedrockRuntimeClient({});
const sv = new S3VectorsClient({});
const BATCH = 200;   // verify the service max per put_vectors call at build time (2026 API)
 
async function embed(text: string): Promise<number[]> {
  const r = await bedrock.send(new InvokeModelCommand({
    modelId: "amazon.titan-embed-text-v2:0",
    body: JSON.stringify({ inputText: text }),
  }));
  return JSON.parse(new TextDecoder().decode(r.body)).embedding;
}
 
async function load(docs: { id: string; text: string; topic: string }[]) {
  const vectors = await Promise.all(docs.map(async (d) => ({
    key: d.id,                                    // stable key -> upsert on re-run
    data: { float32: await embed(d.text) },
    metadata: { topic: d.topic, raw_text: d.text },
  })));
 
  for (let i = 0; i < vectors.length; i += BATCH) {
    const slice = vectors.slice(i, i + BATCH);
    await sv.send(new PutVectorsCommand({ vectorBucketName: "my-vectors", indexName: "docs", vectors: slice }));
  }
}

What this demonstrates:

Embedding is parallelized (a thread pool / Promise.all) because each invoke_model handles one text.
Vectors are grouped into fixed-size batches for put_vectors, not written one by one.
A deterministic key per document makes re-runs upsert instead of duplicating.
Writes retry with exponential backoff so transient throttling does not abort the load.

Deep Dive

Two Bottlenecks, Tuned Separately

A load has two distinct rate limits: how fast you can embed (Bedrock invoke_model throughput and quota) and how fast you can write (put_vectors request rate and batch size). Treat them independently. Parallelize embedding with a worker pool sized to your Bedrock quota, and batch writes to the largest size the service accepts. Confirm that maximum batch size against current docs - it is a 2026 API and the cap may change.

Idempotency and Re-runs

Long loads fail partway. The cure is a stable key: derive each vector's key deterministically from the source item (its id, or a hash of its content), so writing the same item again overwrites rather than duplicates. That turns a crashed job into a safe re-run - reprocess everything and only the missing or changed vectors effectively change.

Checkpointing

For very large corpora, record progress (last processed offset or id) to a file, DynamoDB, or an S3 object. On restart, skip what is already done. Combined with idempotent keys, checkpointing makes a multi-hour, multi-million-vector load robust against interruptions.

Cost Awareness

Two meters run during a load: Bedrock charges per embedded token, and S3 Vectors charges per write request and for storage. Batching cuts the request count (and thus write cost); it does not change embedding cost, which is driven by total tokens. Estimate both before a large backfill so the bill is not a surprise.

Streaming Top-ups

The same loader serves incremental ingestion. Trigger it from S3 event notifications or a queue as new documents land, embedding and upserting just the new items. Because keys are stable, replays and duplicate events are harmless.

Gotchas

One vector per request - single-item writes waste time and money at scale. Fix: batch up to the service maximum per put_vectors call.
Random keys - UUIDs generated per run make re-runs duplicate every vector. Fix: derive the key from the source id or a content hash.
Unbounded embedding concurrency - too many parallel invoke_model calls hit Bedrock throttling. Fix: size the worker pool to your model quota and back off on 429.
No retries - a transient throttle aborts the whole load. Fix: wrap writes (and embeds) in retry-with-backoff.
Dimension drift - mixing embeddings of different lengths fails the write. Fix: pin one model and assert len(vec) == dimension before batching.
Ignoring partial-batch errors - a batch response may report per-vector failures. Fix: inspect the response and retry only the failed keys, which idempotent keys make safe.

Alternatives

Alternative	Use When	Don't Use When
Batched `put_vectors` loader (this page)	You manage embeddings and want control	A managed pipeline would suffice
Bedrock Knowledge Base ingestion	You want managed chunk/embed/load	You need custom chunking or keys
Streaming top-up via events	New documents arrive continuously	A one-time bulk backfill
Offline embed then bulk load	Embeddings already exist in a file	You must embed fresh each run
Single `put_vectors` per item	Tiny datasets or one-off inserts	Thousands or millions of vectors

FAQs

How many vectors can one put_vectors call take?

It accepts a list, so batch as many as the service allows per request. The exact maximum is part of a 2026 API - check current docs and size your batch to it (a few hundred is a safe starting point).

How do I avoid duplicating vectors on a re-run?

Use a deterministic key per source item - its id or a content hash. Writing the same key upserts, so reprocessing the corpus overwrites in place instead of creating duplicates.

Should I embed and write in the same loop?

Separate their concurrency. Embedding is bounded by Bedrock quota and best parallelized with a worker pool; writing is bounded by request rate and best batched. Tuning them independently maximizes throughput.

How do I make a huge load resumable?

Combine idempotent keys with checkpointing - persist the last processed id or offset, and on restart skip completed items. A crash then costs a restart, not a corrupt index.

What throttles a large load?

Usually Bedrock embedding quota first, then the put_vectors request rate. Add backoff-and-retry on both, and size embedding concurrency to your model's limit to avoid 429s.

Can I load embeddings I generated elsewhere?

Yes. If you already have float arrays (from a file or another service), skip the embed step and batch them straight into put_vectors, as long as their length matches the index dimension.

How do I do incremental updates?

Run the same loader on just the new or changed items, triggered by S3 events or a queue. Stable keys make replays and duplicate deliveries safe, so at-least-once delivery is fine.

What does batching actually save?

Request count, and therefore write cost and wall-clock time. It does not lower embedding cost, which depends on total tokens processed, so estimate that separately for big backfills.

S3 Vectors Basics - the single-write building block this pipeline scales up.
Vector Indexes & Similarity Search Queries - querying the index once it is loaded.
Integrating S3 Vectors with Bedrock Knowledge Bases - let Bedrock own ingestion instead.
S3 Vectors via SDK Best Practices - cost and reliability habits for loading and querying.

Stack versions: This page was written for boto3 1.43.x (Python 3.10+) and the AWS SDK for JavaScript v3 (Node.js 18+). Amazon S3 Vectors is a fast-moving 2026 API - verify exact operation and parameter names against current docs.

Recipe

Quick-reference recipe card - batch many vectors into one write.

# --- Python (boto3) ---
import boto3
 
sv = boto3.client("s3vectors")
 
# Group vectors and write them in one call instead of one-per-request.
batch = [
    {"key": doc_id, "data": {"float32": vec}, "metadata": meta}
    for doc_id, vec, meta in rows
]
sv.put_vectors(vectorBucketName="my-vectors", indexName="docs", vectors=batch)
# Confirm the max vectors per put_vectors call against current boto3 docs (2026 API surface).

// --- TypeScript (AWS SDK v3) ---
import { S3VectorsClient, PutVectorsCommand } from "@aws-sdk/client-s3vectors";
 
const sv = new S3VectorsClient({});
 
const batch = rows.map(({ docId, vec, meta }) => ({
  key: docId, data: { float32: vec }, metadata: meta,
}));
await sv.send(new PutVectorsCommand({ vectorBucketName: "my-vectors", indexName: "docs", vectors: batch }));
// Confirm the max vectors per put_vectors call against current @aws-sdk/client-s3vectors docs (2026 API surface).

When to reach for this:

Initial load of an existing document corpus into a new index.
Nightly or streaming top-ups as new documents arrive.
Re-embedding after switching models (a new index plus a full reload).
Backfilling metadata or re-keying an existing dataset.

Working Example

An end-to-end batch loader: chunked embedding with bounded concurrency, batched writes, retries, and a stable key so re-runs upsert. Shown in Python with a smaller illustrative TypeScript equivalent.

# --- Python (boto3) ---
import boto3, json, time
from concurrent.futures import ThreadPoolExecutor
 
bedrock = boto3.client("bedrock-runtime")
sv = boto3.client("s3vectors")
 
BATCH = 200   # vectors per put_vectors call - verify the service max at build time (2026 API)
 
def embed(text: str) -> list[float]:
    resp = bedrock.invoke_model(
        modelId="amazon.titan-embed-text-v2:0",
        body=json.dumps({"inputText": text}),
    )
    return json.loads(resp["body"].read())["embedding"]
 
def put_with_retry(vectors, attempts=5):
    for i in range(attempts):
        try:
            sv.put_vectors(vectorBucketName="my-vectors", indexName="docs", vectors=vectors)
            return
        except Exception:
            if i == attempts - 1:
                raise
            time.sleep(2 ** i)   # exponential backoff
 
def load(docs):   # docs: list[{"id":..., "text":..., "topic":...}]
    with ThreadPoolExecutor(max_workers=8) as pool:
        embeddings = list(pool.map(lambda d: embed(d["text"]), docs))
 
    batch = []
    for d, vec in zip(docs, embeddings):
        batch.append({
            "key": d["id"],                                   # stable key -> upsert on re-run
            "data": {"float32": vec},
            "metadata": {"topic": d["topic"], "raw_text": d["text"]},
        })
        if len(batch) >= BATCH:
            put_with_retry(batch)
            batch = []
    if batch:
        put_with_retry(batch)
 
# load(read_corpus())  # feed your source iterator

// --- TypeScript (AWS SDK v3) ---
import { BedrockRuntimeClient, InvokeModelCommand } from "@aws-sdk/client-bedrock-runtime";
import { S3VectorsClient, PutVectorsCommand } from "@aws-sdk/client-s3vectors";
 
const bedrock = new BedrockRuntimeClient({});
const sv = new S3VectorsClient({});
const BATCH = 200;   // verify the service max per put_vectors call at build time (2026 API)
 
async function embed(text: string): Promise<number[]> {
  const r = await bedrock.send(new InvokeModelCommand({
    modelId: "amazon.titan-embed-text-v2:0",
    body: JSON.stringify({ inputText: text }),
  }));
  return JSON.parse(new TextDecoder().decode(r.body)).embedding;
}
 
async function load(docs: { id: string; text: string; topic: string }[]) {
  const vectors = await Promise.all(docs.map(async (d) => ({
    key: d.id,                                    // stable key -> upsert on re-run
    data: { float32: await embed(d.text) },
    metadata: { topic: d.topic, raw_text: d.text },
  })));
 
  for (let i = 0; i < vectors.length; i += BATCH) {
    const slice = vectors.slice(i, i + BATCH);
    await sv.send(new PutVectorsCommand({ vectorBucketName: "my-vectors", indexName: "docs", vectors: slice }));
  }
}

What this demonstrates:

Embedding is parallelized (a thread pool / Promise.all) because each invoke_model handles one text.
Vectors are grouped into fixed-size batches for put_vectors, not written one by one.
A deterministic key per document makes re-runs upsert instead of duplicating.
Writes retry with exponential backoff so transient throttling does not abort the load.

Deep Dive

Two Bottlenecks, Tuned Separately

Idempotency and Re-runs

Checkpointing

Cost Awareness

Streaming Top-ups

Gotchas

One vector per request - single-item writes waste time and money at scale. Fix: batch up to the service maximum per put_vectors call.
Random keys - UUIDs generated per run make re-runs duplicate every vector. Fix: derive the key from the source id or a content hash.
Unbounded embedding concurrency - too many parallel invoke_model calls hit Bedrock throttling. Fix: size the worker pool to your model quota and back off on 429.
No retries - a transient throttle aborts the whole load. Fix: wrap writes (and embeds) in retry-with-backoff.
Dimension drift - mixing embeddings of different lengths fails the write. Fix: pin one model and assert len(vec) == dimension before batching.
Ignoring partial-batch errors - a batch response may report per-vector failures. Fix: inspect the response and retry only the failed keys, which idempotent keys make safe.

Alternatives

Alternative	Use When	Don't Use When
Batched `put_vectors` loader (this page)	You manage embeddings and want control	A managed pipeline would suffice
Bedrock Knowledge Base ingestion	You want managed chunk/embed/load	You need custom chunking or keys
Streaming top-up via events	New documents arrive continuously	A one-time bulk backfill
Offline embed then bulk load	Embeddings already exist in a file	You must embed fresh each run
Single `put_vectors` per item	Tiny datasets or one-off inserts	Thousands or millions of vectors

FAQs

How many vectors can one put_vectors call take?

It accepts a list, so batch as many as the service allows per request. The exact maximum is part of a 2026 API - check current docs and size your batch to it (a few hundred is a safe starting point).

How do I avoid duplicating vectors on a re-run?

Use a deterministic key per source item - its id or a content hash. Writing the same key upserts, so reprocessing the corpus overwrites in place instead of creating duplicates.

Should I embed and write in the same loop?

How do I make a huge load resumable?

Combine idempotent keys with checkpointing - persist the last processed id or offset, and on restart skip completed items. A crash then costs a restart, not a corrupt index.

What throttles a large load?

Usually Bedrock embedding quota first, then the put_vectors request rate. Add backoff-and-retry on both, and size embedding concurrency to your model's limit to avoid 429s.

Can I load embeddings I generated elsewhere?

Yes. If you already have float arrays (from a file or another service), skip the embed step and batch them straight into put_vectors, as long as their length matches the index dimension.

How do I do incremental updates?

Run the same loader on just the new or changed items, triggered by S3 events or a queue. Stable keys make replays and duplicate deliveries safe, so at-least-once delivery is fine.

What does batching actually save?

Request count, and therefore write cost and wall-clock time. It does not lower embedding cost, which depends on total tokens processed, so estimate that separately for big backfills.

S3 Vectors Basics - the single-write building block this pipeline scales up.
Vector Indexes & Similarity Search Queries - querying the index once it is loaded.
Integrating S3 Vectors with Bedrock Knowledge Bases - let Bedrock own ingestion instead.
S3 Vectors via SDK Best Practices - cost and reliability habits for loading and querying.

Stack versions: This page was written for boto3 1.43.x (Python 3.10+) and the AWS SDK for JavaScript v3 (Node.js 18+). Amazon S3 Vectors is a fast-moving 2026 API - verify exact operation and parameter names against current docs.

Batch-Loading Embeddings at Scale

Buscar na documentação

Batch-Loading Embeddings at Scale