Streaming Responses with ConverseStream

Token-by-token streaming for responsive UIs via SDK.

ConverseStream is the streaming twin of Converse. Instead of waiting for the whole reply and returning one object, it returns a stream of events you consume as the model generates. That makes UIs feel instant - text appears as it is produced. This page walks the full event lifecycle, shows how to assemble the text, and reads the final usage event for billing.

Recipe

Quick-reference recipe card - copy-paste ready.

# --- Python (boto3) ---
import boto3
 
brt = boto3.client("bedrock-runtime", region_name="us-east-1")
resp = brt.converse_stream(
    modelId="amazon.nova-lite-v1:0",  # representative id - verify per region
    messages=[{"role": "user", "content": [{"text": "Explain SQS in two sentences."}]}],
)
for event in resp["stream"]:
    if "contentBlockDelta" in event:
        print(event["contentBlockDelta"]["delta"]["text"], end="", flush=True)

// --- TypeScript (AWS SDK v3) ---
import { BedrockRuntimeClient, ConverseStreamCommand } from "@aws-sdk/client-bedrock-runtime";
 
const brt = new BedrockRuntimeClient({ region: "us-east-1" });
const resp = await brt.send(new ConverseStreamCommand({
  modelId: "amazon.nova-lite-v1:0", // representative id - verify per region
  messages: [{ role: "user", content: [{ text: "Explain SQS in two sentences." }] }],
}));
for await (const event of resp.stream!) {
  if (event.contentBlockDelta) process.stdout.write(event.contentBlockDelta.delta?.text ?? "");
}

When to reach for this:

• Chat UIs and assistants where users should see output immediately.
• Long generations where waiting for the full response feels slow.
• Any front end that can render partial text as it arrives.
• Skip it for batch jobs where you only need the final string - plain Converse is simpler.

Working Example

Consume the full lifecycle: accumulate the text from deltas, then read the terminal messageStop and metadata events for the stop reason and token usage.

# --- Python (boto3) ---
import boto3
 
brt = boto3.client("bedrock-runtime", region_name="us-east-1")
resp = brt.converse_stream(
    modelId="amazon.nova-lite-v1:0",
    messages=[{"role": "user", "content": [{"text": "List three uses for S3."}]}],
    inferenceConfig={"maxTokens": 300},
)
 
text = ""
for event in resp["stream"]:
    if "messageStart" in event:
        role = event["messageStart"]["role"]          # "assistant"
    elif "contentBlockDelta" in event:
        chunk = event["contentBlockDelta"]["delta"].get("text", "")
        text += chunk
        print(chunk, end="", flush=True)
    elif "messageStop" in event:
        stop = event["messageStop"]["stopReason"]      # end_turn / max_tokens / ...
    elif "metadata" in event:
        usage = event["metadata"]["usage"]             # inputTokens / outputTokens / totalTokens
 
print("\n---\nstop:", stop, "| tokens:", usage["totalTokens"])

// --- TypeScript (AWS SDK v3) ---
import { BedrockRuntimeClient, ConverseStreamCommand } from "@aws-sdk/client-bedrock-runtime";
 
const brt = new BedrockRuntimeClient({ region: "us-east-1" });
const resp = await brt.send(new ConverseStreamCommand({
  modelId: "amazon.nova-lite-v1:0",
  messages: [{ role: "user", content: [{ text: "List three uses for S3." }] }],
  inferenceConfig: { maxTokens: 300 },
}));
 
let text = "";
let stop: string | undefined;
let usage;
for await (const event of resp.stream!) {
  if (event.messageStart) {
    // role === "assistant"
  } else if (event.contentBlockDelta) {
    const chunk = event.contentBlockDelta.delta?.text ?? "";
    text += chunk;
    process.stdout.write(chunk);
  } else if (event.messageStop) {
    stop = event.messageStop.stopReason;
  } else if (event.metadata) {
    usage = event.metadata.usage; // inputTokens / outputTokens / totalTokens
  }
}
console.log(`\n---\nstop: ${stop} | tokens: ${usage?.totalTokens}`);

What this demonstrates:

• The stream is a sequence of typed events - you branch on which member is present.
• Text lives only in contentBlockDelta events; accumulate them for the complete reply.
• messageStop carries the stopReason, exactly like the non-streaming response.
• The final metadata event carries usage, so you still get token counts for billing.

Deep Dive

The Event Lifecycle

A ConverseStream response is an ordered event stream. For a simple text reply the sequence is:

1. messageStart - announces the assistant turn and its role.
2. contentBlockStart - marks the beginning of a content block (index and type; important for tools).
3. contentBlockDelta - the incremental payload; for text, delta.text.
4. contentBlockStop - closes that content block.
5. messageStop - ends the message and reports stopReason.
6. metadata - final event with usage (token counts) and metrics (latency).

For plain text you mostly care about contentBlockDelta, messageStop, and metadata. The contentBlockStart/Stop pairs matter when a response has multiple blocks - for example a text block plus a tool-use block.

Streaming Tool Use

When a streamed reply calls a tool, the tool arguments arrive incrementally too. The contentBlockStart event names the toolUse (id and name), and successive contentBlockDelta events deliver the input JSON as a growing string you concatenate and parse once the block stops. The stopReason then comes back as tool_use. The mechanics of responding to a tool call are covered on the tool-use page; streaming just changes how you receive the request.

Errors Arrive as Events

Because the connection is already open, some failures surface mid-stream rather than as an exception on the initial call. boto3 raises modeled exceptions such as ModelStreamErrorException or ThrottlingException while you iterate; in SDK v3 the async iterator rejects. Wrap the iteration in try/except (Python) or try/catch (TypeScript) so a partial stream fails cleanly, and consider flushing what you have already rendered.

Cost and Latency

Streaming changes when you see tokens, not how many you pay for. The usage totals in the final metadata event match what a non-streaming Converse call would report for the same generation. What you gain is perceived latency: time-to-first-token is far shorter than time-to-full-response, which is the whole point for interactive UIs.

Gotchas

• Treating an event as always having text - only contentBlockDelta events carry delta.text; others do not. Fix: branch on which member is present before reading text.
• Missing the final usage - usage is in the terminal metadata event, not on each delta. Fix: capture metadata.usage at the end if you need token counts.
• Not handling mid-stream errors - throttling or stream errors can appear while iterating. Fix: wrap the loop in try/except (boto3) or try/catch (v3) and handle partial output.
• Assuming one content block - tool use and multi-block replies interleave contentBlockStart/Stop. Fix: track the block index, do not assume a single text block.
• Buffering the whole stream before rendering - that throws away the latency benefit. Fix: render each delta as it arrives.
• Forgetting streaming needs its own IAM action in some setups - bedrock:InvokeModelWithResponseStream may be required. Fix: grant the streaming permission alongside bedrock:InvokeModel.

Alternatives

FAQs

Related

• Bedrock Basics - the non-streaming Converse call this builds on.
• The Converse API vs InvokeModel - streaming exists on both surfaces.
• Tool Use / Function Calling via Bedrock - how streamed tool calls are handled.
• Bedrock via SDK Best Practices - retries, timeouts, and cost monitoring for streams.

Stack versions: This page was written for boto3 1.43.x (Python 3.10+) and the AWS SDK for JavaScript v3 (Node.js 18+). Bedrock model IDs and inference profiles are region/version-specific - verify against the current Bedrock catalog at build time.