Polly Voices, SSML & Speech Marks

Q: Do I need SSML to use Polly at all?

No. Plain text (TextType: "text", the default) works for simple cases. SSML is opt-in, needed only when you want pacing, emphasis, or pronunciation control beyond what plain text gives you.

Q: Can I get speech marks and audio in one call?

No. OutputFormat is either an audio format or "json" for marks, not both at once. Getting synced captions requires two separate SynthesizeSpeech calls with matching text and voice settings.

Q: Which engine should I default to?

neural for most user-facing prompts where natural sound matters; standard where cost matters more than voice quality; long-form specifically for longer narration-style content. Check DescribeVoices to confirm your chosen voice supports the engine you want.

Q: Can every SSML tag be used with every Polly voice?

Not necessarily - tag support can differ by engine, with neural generally supporting a narrower set than standard. Check current Polly documentation for the tag support matrix before relying on a less common tag.

SynthesizeSpeech covers the basic case - text in, audio out - but three more controls shape what that audio actually sounds like and how well it integrates with the rest of your application: which voice and engine to use, how to mark up the text with SSML, and how to request timing metadata via speech marks.

This page covers all three, building on the plain synthesis call shown earlier in this section.

Recipe

Quick-reference recipe card - copy-paste ready.

# --- Python (boto3) ---
import boto3
 
polly = boto3.client("polly")
response = polly.synthesize_speech(
    Text='<speak>Welcome back. <break time="200ms"/> You have <emphasis level="strong">3</emphasis> new messages.</speak>',
    TextType="ssml",
    OutputFormat="mp3",
    VoiceId="Joanna",
    Engine="neural",
)
with open("welcome.mp3", "wb") as f:
    f.write(response["AudioStream"].read())

// --- TypeScript (AWS SDK v3) ---
import { PollyClient, SynthesizeSpeechCommand } from "@aws-sdk/client-polly";
import { writeFileSync } from "node:fs";
 
const polly = new PollyClient({});
const response = await polly.send(new SynthesizeSpeechCommand({
  Text: '<speak>Welcome back. <break time="200ms"/> You have <emphasis level="strong">3</emphasis> new messages.</speak>',
  TextType: "ssml",
  OutputFormat: "mp3",
  VoiceId: "Joanna",
  Engine: "neural",
}));
const bytes = await response.AudioStream?.transformToByteArray();
writeFileSync("welcome.mp3", Buffer.from(bytes ?? []));

When to reach for this:

Any prompt where pacing or emphasis matters - IVR menus, notifications, alerts.
Choosing a voice deliberately rather than hardcoding the first one you tried.
Building synced captions or mouth animation alongside playback.
Comparing engine cost/quality trade-offs before committing to one for a high-volume feature.

Working Example

List available voices for a language, synthesize SSML-marked-up speech with a chosen voice and engine, then request speech marks for the same text to get word-level timing.

# --- Python (boto3) ---
import json
import boto3
 
polly = boto3.client("polly")
 
# 1. Find neural-capable voices for English.
voices = polly.describe_voices(LanguageCode="en-US", Engine="neural")
voice_id = voices["Voices"][0]["Id"]
 
ssml = '<speak>Your total is <break time="300ms"/> <say-as interpret-as="currency">USD42.00</say-as>.</speak>'
 
# 2. Synthesize the audio.
audio = polly.synthesize_speech(
    Text=ssml, TextType="ssml", OutputFormat="mp3", VoiceId=voice_id, Engine="neural",
)
with open("total.mp3", "wb") as f:
    f.write(audio["AudioStream"].read())
 
# 3. Request word-level timing for the same text.
marks = polly.synthesize_speech(
    Text=ssml,
    TextType="ssml",
    OutputFormat="json",
    VoiceId=voice_id,
    Engine="neural",
    SpeechMarkTypes=["word"],
)
for line in marks["AudioStream"].read().decode("utf-8").splitlines():
    mark = json.loads(line)
    print(mark["time"], mark["value"])

// --- TypeScript (AWS SDK v3) ---
import { PollyClient, SynthesizeSpeechCommand, DescribeVoicesCommand } from "@aws-sdk/client-polly";
import { writeFileSync } from "node:fs";
 
const polly = new PollyClient({});
 
// 1. Find neural-capable voices for English.
const voices = await polly.send(new DescribeVoicesCommand({ LanguageCode: "en-US", Engine: "neural" }));
const voiceId = voices.Voices?.[0]?.Id ?? "Joanna";
 
const ssml = '<speak>Your total is <break time="300ms"/> <say-as interpret-as="currency">USD42.00</say-as>.</speak>';
 
// 2. Synthesize the audio.
const audio = await polly.send(new SynthesizeSpeechCommand({
  Text: ssml, TextType: "ssml", OutputFormat: "mp3", VoiceId: voiceId, Engine: "neural",
}));
const audioBytes = await audio.AudioStream?.transformToByteArray();
writeFileSync("total.mp3", Buffer.from(audioBytes ?? []));
 
// 3. Request word-level timing for the same text.
const marks = await polly.send(new SynthesizeSpeechCommand({
  Text: ssml,
  TextType: "ssml",
  OutputFormat: "json",
  VoiceId: voiceId,
  Engine: "neural",
  SpeechMarkTypes: ["word"],
}));
const marksText = await marks.AudioStream?.transformToString();
for (const line of (marksText ?? "").split("\n").filter(Boolean)) {
  const mark = JSON.parse(line);
  console.log(mark.time, mark.value);
}

What this demonstrates:

DescribeVoices with Engine: "neural" filters to only voices that support that engine - not every voice does.
<say-as interpret-as="currency"> is an SSML tag that tells Polly to read a value as currency rather than as literal digits.
Requesting OutputFormat: "json" with SpeechMarkTypes returns newline-delimited JSON timing events instead of audio - a second, separate call from the one that produces the MP3.
Each speech mark carries a time (milliseconds into the audio) and a value (the word or tag it corresponds to), which is exactly what you need to sync captions or animation to playback.

Deep Dive

Choosing a Voice and Engine

DescribeVoices is the source of truth for what's available - voice IDs, supported languages, and which Engine values (standard, neural, long-form) each voice supports. Not every voice supports every engine, and coverage changes as AWS adds voices, so query it rather than hardcoding a voice/engine pair you haven't verified still exists.

standard is the original, lower-cost engine. neural produces markedly more natural, human-like speech at a higher per-character price and with narrower regional availability. long-form is tuned specifically for longer-form content like articles or audiobook-style narration, prioritizing consistency over a long passage. Pick based on the actual use case rather than defaulting to the most expensive option everywhere.

SSML Tags in Practice

Setting TextType: "ssml" switches Polly from reading Text literally to interpreting it as markup, and the payload must be wrapped in a single <speak> root element. Commonly used tags:

<break time="300ms"/> - inserts a pause of a specific duration.
<emphasis level="strong">...</emphasis> - stresses a word or phrase.
<prosody rate="slow" pitch="+10%">...</prosody> - adjusts speaking rate and pitch over a span.
<say-as interpret-as="currency">USD42.00</say-as> - controls how a value is read aloud (currency, date, telephone number, and other interpretations).
<phoneme alphabet="ipa" ph="...">...</phoneme> - specifies exact pronunciation for a word Polly would otherwise mispronounce.

Not every SSML tag is supported by every engine - neural supports a narrower tag set than standard. Check current Polly documentation for the exact tag support matrix per engine before relying on a less common tag in a neural voice.

Speech Marks Are Metadata, Not Sound

Requesting OutputFormat: "json" (instead of mp3, ogg_vorbis, or pcm) changes what the call returns entirely - a stream of newline-delimited JSON objects, not audio bytes. Each object represents one mark: a time offset in milliseconds, a type (word, sentence, ssml, or viseme), and a value (the text, or for viseme, a code representing a mouth-shape for animation).

Because this is a separate call from the one producing audio, the typical pattern is exactly what the working example shows: call SynthesizeSpeech once for the audio, and again with the same Text/TextType but OutputFormat: "json" and SpeechMarkTypes set, for the timing. The two calls must use matching text and voice settings for the marks to line up correctly with the audio you'll play them against.

Speech Mark Types and Their Use Cases

word - per-word timing, the most common choice for synced captions.
sentence - per-sentence timing, useful for coarser caption chunks.
ssml - timing for specific SSML markers you place yourself, useful for cueing custom events (like highlighting a specific UI element) at an exact point in the audio.
viseme - mouth-shape codes for lip-sync animation, one of the more specialized uses of this feature.

Gotchas

Forgetting TextType: "ssml". Without it, markup tags are read aloud literally as text instead of being interpreted. Fix: always set TextType: "ssml" when the Text payload contains SSML tags.
Omitting the <speak> root tag. SSML input without a single wrapping <speak> element is rejected. Fix: always wrap the full SSML payload in one <speak>...</speak>.
Assuming every voice supports every engine. A voice ID that works with standard may not exist for neural or long-form. Fix: check SupportedEngines from DescribeVoices before hardcoding a voice/engine pair.
Expecting OutputFormat: "json" to return audio. It returns speech mark metadata only - no sound. Fix: make a separate SynthesizeSpeech call with an audio OutputFormat (mp3, ogg_vorbis, pcm) for the actual playback file.
Mismatching the text between the audio call and the marks call. If the two calls use different text, voice, or engine, timings won't line up with the audio you actually play. Fix: keep Text, TextType, VoiceId, and Engine identical across both calls.
Assuming an SSML tag works the same on every engine. Some tags supported on standard are unsupported or behave differently on neural. Fix: verify tag support against the engine you're actually using, not just the SSML spec in general.

Alternatives

Alternative	Use When	Don't Use When
Plain text synthesis	Simple prompts with no pacing or emphasis needs	You need precise pacing, pronunciation, or emphasis control
SSML (this page)	Controlling pacing, emphasis, or pronunciation	Plain text already sounds correct as-is
Speech marks (this page)	Syncing captions or animation to audio playback	You only need the audio itself, no timing metadata
standard engine	Cost-sensitive, high-volume synthesis where quality bar is lower	Natural-sounding speech is a priority
neural / long-form engine	Natural-sounding prompts, narration, long-form content	Budget is the primary constraint

FAQs

Do I need SSML to use Polly at all?

No. Plain text (TextType: "text", the default) works for simple cases. SSML is opt-in, needed only when you want pacing, emphasis, or pronunciation control beyond what plain text gives you.

What happens if I set TextType to ssml but the text has no valid SSML tags?

Text without any markup is still valid SSML as long as it's wrapped in <speak>...</speak> - Polly reads it normally. The root <speak> tag itself is what's required.

Are speech marks audio?

No. They're JSON timing metadata - word, sentence, SSML marker, or viseme events with millisecond offsets - returned instead of audio when you set OutputFormat: "json".

Can I get speech marks and audio in one call?

No. OutputFormat is either an audio format or "json" for marks, not both at once. Getting synced captions requires two separate SynthesizeSpeech calls with matching text and voice settings.

Which engine should I default to?

neural for most user-facing prompts where natural sound matters; standard where cost matters more than voice quality; long-form specifically for longer narration-style content. Check DescribeVoices to confirm your chosen voice supports the engine you want.

Can every SSML tag be used with every Polly voice?

Not necessarily - tag support can differ by engine, with neural generally supporting a narrower set than standard. Check current Polly documentation for the tag support matrix before relying on a less common tag.

Speech Services Basics - the first plain-text SynthesizeSpeech call.
Transcribe & Polly: Speech In, Speech Out - how Polly fits opposite Transcribe.
Speech Services via SDK Best Practices - cost-aware engine and format choices.

Stack versions: This page was written for boto3 1.43.x (Python 3.10+) and the AWS SDK for JavaScript v3 (Node.js 18+).

This page covers all three, building on the plain synthesis call shown earlier in this section.

Recipe

Quick-reference recipe card - copy-paste ready.

# --- Python (boto3) ---
import boto3
 
polly = boto3.client("polly")
response = polly.synthesize_speech(
    Text='<speak>Welcome back. <break time="200ms"/> You have <emphasis level="strong">3</emphasis> new messages.</speak>',
    TextType="ssml",
    OutputFormat="mp3",
    VoiceId="Joanna",
    Engine="neural",
)
with open("welcome.mp3", "wb") as f:
    f.write(response["AudioStream"].read())

// --- TypeScript (AWS SDK v3) ---
import { PollyClient, SynthesizeSpeechCommand } from "@aws-sdk/client-polly";
import { writeFileSync } from "node:fs";
 
const polly = new PollyClient({});
const response = await polly.send(new SynthesizeSpeechCommand({
  Text: '<speak>Welcome back. <break time="200ms"/> You have <emphasis level="strong">3</emphasis> new messages.</speak>',
  TextType: "ssml",
  OutputFormat: "mp3",
  VoiceId: "Joanna",
  Engine: "neural",
}));
const bytes = await response.AudioStream?.transformToByteArray();
writeFileSync("welcome.mp3", Buffer.from(bytes ?? []));

When to reach for this:

Any prompt where pacing or emphasis matters - IVR menus, notifications, alerts.
Choosing a voice deliberately rather than hardcoding the first one you tried.
Building synced captions or mouth animation alongside playback.
Comparing engine cost/quality trade-offs before committing to one for a high-volume feature.

Working Example

List available voices for a language, synthesize SSML-marked-up speech with a chosen voice and engine, then request speech marks for the same text to get word-level timing.

# --- Python (boto3) ---
import json
import boto3
 
polly = boto3.client("polly")
 
# 1. Find neural-capable voices for English.
voices = polly.describe_voices(LanguageCode="en-US", Engine="neural")
voice_id = voices["Voices"][0]["Id"]
 
ssml = '<speak>Your total is <break time="300ms"/> <say-as interpret-as="currency">USD42.00</say-as>.</speak>'
 
# 2. Synthesize the audio.
audio = polly.synthesize_speech(
    Text=ssml, TextType="ssml", OutputFormat="mp3", VoiceId=voice_id, Engine="neural",
)
with open("total.mp3", "wb") as f:
    f.write(audio["AudioStream"].read())
 
# 3. Request word-level timing for the same text.
marks = polly.synthesize_speech(
    Text=ssml,
    TextType="ssml",
    OutputFormat="json",
    VoiceId=voice_id,
    Engine="neural",
    SpeechMarkTypes=["word"],
)
for line in marks["AudioStream"].read().decode("utf-8").splitlines():
    mark = json.loads(line)
    print(mark["time"], mark["value"])

// --- TypeScript (AWS SDK v3) ---
import { PollyClient, SynthesizeSpeechCommand, DescribeVoicesCommand } from "@aws-sdk/client-polly";
import { writeFileSync } from "node:fs";
 
const polly = new PollyClient({});
 
// 1. Find neural-capable voices for English.
const voices = await polly.send(new DescribeVoicesCommand({ LanguageCode: "en-US", Engine: "neural" }));
const voiceId = voices.Voices?.[0]?.Id ?? "Joanna";
 
const ssml = '<speak>Your total is <break time="300ms"/> <say-as interpret-as="currency">USD42.00</say-as>.</speak>';
 
// 2. Synthesize the audio.
const audio = await polly.send(new SynthesizeSpeechCommand({
  Text: ssml, TextType: "ssml", OutputFormat: "mp3", VoiceId: voiceId, Engine: "neural",
}));
const audioBytes = await audio.AudioStream?.transformToByteArray();
writeFileSync("total.mp3", Buffer.from(audioBytes ?? []));
 
// 3. Request word-level timing for the same text.
const marks = await polly.send(new SynthesizeSpeechCommand({
  Text: ssml,
  TextType: "ssml",
  OutputFormat: "json",
  VoiceId: voiceId,
  Engine: "neural",
  SpeechMarkTypes: ["word"],
}));
const marksText = await marks.AudioStream?.transformToString();
for (const line of (marksText ?? "").split("\n").filter(Boolean)) {
  const mark = JSON.parse(line);
  console.log(mark.time, mark.value);
}

What this demonstrates:

DescribeVoices with Engine: "neural" filters to only voices that support that engine - not every voice does.
<say-as interpret-as="currency"> is an SSML tag that tells Polly to read a value as currency rather than as literal digits.
Requesting OutputFormat: "json" with SpeechMarkTypes returns newline-delimited JSON timing events instead of audio - a second, separate call from the one that produces the MP3.
Each speech mark carries a time (milliseconds into the audio) and a value (the word or tag it corresponds to), which is exactly what you need to sync captions or animation to playback.

Deep Dive

Choosing a Voice and Engine

SSML Tags in Practice

Setting TextType: "ssml" switches Polly from reading Text literally to interpreting it as markup, and the payload must be wrapped in a single <speak> root element. Commonly used tags:

<break time="300ms"/> - inserts a pause of a specific duration.
<emphasis level="strong">...</emphasis> - stresses a word or phrase.
<prosody rate="slow" pitch="+10%">...</prosody> - adjusts speaking rate and pitch over a span.
<say-as interpret-as="currency">USD42.00</say-as> - controls how a value is read aloud (currency, date, telephone number, and other interpretations).
<phoneme alphabet="ipa" ph="...">...</phoneme> - specifies exact pronunciation for a word Polly would otherwise mispronounce.

Speech Marks Are Metadata, Not Sound

Speech Mark Types and Their Use Cases

word - per-word timing, the most common choice for synced captions.
sentence - per-sentence timing, useful for coarser caption chunks.
ssml - timing for specific SSML markers you place yourself, useful for cueing custom events (like highlighting a specific UI element) at an exact point in the audio.
viseme - mouth-shape codes for lip-sync animation, one of the more specialized uses of this feature.

Gotchas

Forgetting TextType: "ssml". Without it, markup tags are read aloud literally as text instead of being interpreted. Fix: always set TextType: "ssml" when the Text payload contains SSML tags.
Omitting the <speak> root tag. SSML input without a single wrapping <speak> element is rejected. Fix: always wrap the full SSML payload in one <speak>...</speak>.
Assuming every voice supports every engine. A voice ID that works with standard may not exist for neural or long-form. Fix: check SupportedEngines from DescribeVoices before hardcoding a voice/engine pair.
Expecting OutputFormat: "json" to return audio. It returns speech mark metadata only - no sound. Fix: make a separate SynthesizeSpeech call with an audio OutputFormat (mp3, ogg_vorbis, pcm) for the actual playback file.
Mismatching the text between the audio call and the marks call. If the two calls use different text, voice, or engine, timings won't line up with the audio you actually play. Fix: keep Text, TextType, VoiceId, and Engine identical across both calls.
Assuming an SSML tag works the same on every engine. Some tags supported on standard are unsupported or behave differently on neural. Fix: verify tag support against the engine you're actually using, not just the SSML spec in general.

Alternatives

Alternative	Use When	Don't Use When
Plain text synthesis	Simple prompts with no pacing or emphasis needs	You need precise pacing, pronunciation, or emphasis control
SSML (this page)	Controlling pacing, emphasis, or pronunciation	Plain text already sounds correct as-is
Speech marks (this page)	Syncing captions or animation to audio playback	You only need the audio itself, no timing metadata
standard engine	Cost-sensitive, high-volume synthesis where quality bar is lower	Natural-sounding speech is a priority
neural / long-form engine	Natural-sounding prompts, narration, long-form content	Budget is the primary constraint

FAQs

Do I need SSML to use Polly at all?

No. Plain text (TextType: "text", the default) works for simple cases. SSML is opt-in, needed only when you want pacing, emphasis, or pronunciation control beyond what plain text gives you.

What happens if I set TextType to ssml but the text has no valid SSML tags?

Text without any markup is still valid SSML as long as it's wrapped in <speak>...</speak> - Polly reads it normally. The root <speak> tag itself is what's required.

Are speech marks audio?

No. They're JSON timing metadata - word, sentence, SSML marker, or viseme events with millisecond offsets - returned instead of audio when you set OutputFormat: "json".

Can I get speech marks and audio in one call?

No. OutputFormat is either an audio format or "json" for marks, not both at once. Getting synced captions requires two separate SynthesizeSpeech calls with matching text and voice settings.

Which engine should I default to?

Can every SSML tag be used with every Polly voice?

Speech Services Basics - the first plain-text SynthesizeSpeech call.
Transcribe & Polly: Speech In, Speech Out - how Polly fits opposite Transcribe.
Speech Services via SDK Best Practices - cost-aware engine and format choices.

Stack versions: This page was written for boto3 1.43.x (Python 3.10+) and the AWS SDK for JavaScript v3 (Node.js 18+).

Polly Voices, SSML & Speech Marks

Search Documentation

Polly Voices, SSML & Speech Marks