Per-word timings for synthesized speech, for highlighting text in sync with playback.
Word boundaries tell you when each word is spoken in the audio being streamed to you. The usual reason to want them is karaoke-style highlighting: following along in the source text while the speech plays.They are on by default on the wire. Pass word_boundaries=false at connect time to turn them off. The SDKs turn them off unless you ask for them; see In the SDKs. On the HTTP API they come from a route of their own; see Over the HTTP API.
The frame
You receive at most one WordBoundaries frame per request (each Flush concludes one request), and it covers the spoken words in that request’s text, however long the request is. This one is for a request whose text began Hello there, friend,:
flush_id is the request the words belong to. It is the same id that the request’s FlushDone carries.
audio_start_ms is where this request’s audio begins on the session’s audio timeline (see below). It is 0 for the first request.
words has one entry per whitespace-delimited token of the request’s text, in the order you sent them. Each entry has four keys:
word: the word as it appeared in your text.
start and end: when the word is spoken, in milliseconds.
confidence: a number from 0 to 1 for that word’s timing.
All times are integer milliseconds and do not depend on the session’s sample_rate or encoding.
What word contains
word is a whitespace-delimited token from your text, exactly as you sent it, with its punctuation still attached: there,, today?, dog.. The entries are one to one with the tokens you get by splitting the request’s text on whitespace, in the same order, so the nth entry is the nth token. Non-ASCII text is returned as is.A token with no sound of its own, such as the dash in Hello - world or the ? in the French ça va ?, still gets an entry. It has start equal to end and a confidence of 0, and sits at the edge of the word next to it: an opening mark such as ¿ or ( at the start of the word after it, any other token at the end of the word before it. Bonjour, ça va ? gives four entries, the last one ? at the end of va. A token in square brackets, such as [laughs] or [pause], also gets a zero-length entry, placed the same way. Such a tag is not an instruction: the model reads the word inside it aloud, yet its entry stays zero-length (see How text is read). Skip zero-length entries when you highlight.A token the model reads as several words, such as $1,234,567, gets one entry that spans all of them, several seconds long in that case. Markup stays in word as you sent it: **Hello** is one entry, timed over the spoken Hello, since markup is not read out. An emoji standing alone between spaces gets a zero-length entry, since emoji are not read out either. Timing follows your text, not the spoken form, so numbers, symbols, URLs and tags can get stretched or zero-length spans with a low confidence. See How text is read.A frame never lists words the audio does not contain. When the audio of a request leaves part of its text unspoken, the request gets no frame at all.
Times are on the session timeline
start and end are positions on the session’s continuous output-audio timeline: the audio delivered since the session began, across every request. They are not relative to the request’s own audio. If the first request produced 2800 ms of audio, the second request’s frame has audio_start_ms: 2800 and its first word starts at 2800 or later.Two consequences:
Do not add audio_start_ms to start.start is already a position in the concatenated session audio. Adding the offset misplaces every highlight after the first request.
Subtract it to get a position inside the request’s own audio.start - audio_start_ms is where the word falls in that request’s Audio frames.
The timeline never rewinds. Every byte of audio delivered to you holds its place on it, including the audio of a request you then cancelled and the short tail that arrives between your Cancel and the server’s Cancelled, so the next request’s audio_start_ms counts audio you may have dropped and never played. That is why the mapping to your player’s clock below anchors on each request rather than on the session.
When it arrives
Each WordBoundaries frame is sent after the FlushDone for the same flush_id, usually within tens of milliseconds. When you have several requests in flight it often lands after the next request’s first Audio frames have already arrived. Match frames to requests by flush_id, never by arrival order. For example, a sequence with three pipelined requests:
Nothing arrives after Termination. Terminate waits for pending word boundaries before it ends the session, so you can send it right after your last Flush and still receive that request’s frame. Text you never flushed is flushed for you by Terminate and gets a frame of its own. The same drain runs when the session reaches the expires_at time announced in Begin.FlushDone means the request’s audio has been delivered to the socket, not that it has finished playing. Synthesis runs several times faster than real time, so when the WordBoundaries frame arrives the whole request is usually buffered on your side and its first words may already be playing. Schedule highlights against your audio clock instead of reacting to each frame as it arrives, and apply a highlight immediately when its time has already passed.Word boundaries do not delay the audio: a request’s first audio arrives as soon with them as without, so turning them off does not make it faster. A request’s frame comes only after all of its audio, though, so a long reply sent as one request gets its timings once its whole audio has been sent. To highlight from the first sentence on, flush each sentence as a request of its own: its timings then arrive right after that sentence’s audio. Each request is spoken on its own, so this adds a short pause between sentences.You can print the frames yourself. The program below sends two requests and prints each FlushDone and WordBoundaries as it arrives, with every word’s position on the session timeline and inside its own request. Set ASSEMBLYAI_API_KEY and run it with the websockets package (14 or newer):
import asyncioimport jsonimport osimport websockets # pip install -U websocketsURL = "wss://tts.us.assemblyai.com/v1/ws?voice_id=aai-kyla&language=english"API_KEY = os.environ["ASSEMBLYAI_API_KEY"]REQUESTS = ["Hello there, friend.", "Pack my box with five dozen liquor jugs."]async def main(): async with websockets.connect(URL, additional_headers={"Authorization": API_KEY}) as ws: first = json.loads(await ws.recv()) if first["type"] == "Error": raise SystemExit(f"{first['error_code']}: {first['error']}") if first["configuration"]["word_boundaries"] is not True: raise SystemExit("this session will not send WordBoundaries") for text in REQUESTS: await ws.send(json.dumps({"type": "Generate", "text": text})) await ws.send(json.dumps({"type": "Flush"})) await ws.send(json.dumps({"type": "Terminate"})) async for raw in ws: message = json.loads(raw) if message["type"] == "FlushDone": print(f"FlushDone flush_id={message['flush_id']} audio_duration_ms={message['audio_duration_ms']}") elif message["type"] == "WordBoundaries": print(f"WordBoundaries flush_id={message['flush_id']} audio_start_ms={message['audio_start_ms']}") for word in message["words"]: in_request = word["start"] - message["audio_start_ms"] print(f" {word['word']:<10} session {word['start']:>5} to {word['end']:>5} ms request +{in_request} ms") elif message["type"] == "Termination": break elif message["type"] == "Error": raise SystemExit(f"{message['error_code']}: {message['error']}")asyncio.run(main())
Mapping timings to playback
A word time is a position in the audio, and your player has its own clock. To turn one into the other, remember the player time at which a request’s first audio buffer was scheduled and offset from there. In a WebAudio client that is the AudioContext time you passed to source.start() for the request’s first Audio frame:
// Fragment: the scheduling side of a WebAudio client.const ctx = new AudioContext(); // plays the session's audioconst anchors = new Map(); // flush_id -> AudioContext time at which that request's // first Audio buffer was scheduled; set it where you enqueue audioconst highlightWord = (word) => console.log("speaking:", word); // your UI hookfunction highlight(message) { const anchor = anchors.get(message.flush_id); if (anchor === undefined) return; // no audio was scheduled for this request for (const word of message.words) { if (word.start === word.end) continue; // nothing is spoken for this token // start is on the session timeline; the anchor is at audio_start_ms on that timeline. const at = anchor + (word.start - message.audio_start_ms) / 1000; const delay = Math.max(0, (at - ctx.currentTime) * 1000); setTimeout(() => highlightWord(word.word), delay); }}
Anchoring on each request is what keeps highlights correct after a Cancel, when the audio you stopped playing still counts on the server’s timeline. If you play everything the server sends as one uninterrupted stream and never cancel, a single anchor at the session’s first sample is enough: highlight at sessionStart + word.start / 1000.Clear pending highlight timers together with your scheduled audio when you cancel or start a new request.
Treat them as best-effort
WordBoundaries frames are best-effort. A request can complete, with its
audio delivered in full, and never get one. Nothing is sent in the frame’s
place, and the audio stream itself is unaffected either way.
You will not get a frame for:
a request you cancelled with Cancel;
a request containing a stretch the server speaks in one piece whose audio runs past about 32 seconds. The server splits text into such pieces at sentence ends, and inside a sentence that runs long (see How text is read), so this is rare. Other requests in the same session still get theirs;
a Flush with nothing to speak, which gets a FlushDone with audio_duration_ms: 0 and no WordBoundaries;
a request whose audio does not cover all of its text;
a request for which the server could not compute timings, for example after a transient problem during that request; the audio itself is unaffected.
A long request needs no splitting for highlighting: the server times it in parts of whole sentences and sends one frame that lists every token of the request, after its FlushDone.Two consequences worth designing around:
Check the Begin frame, not your request. The configuration echoed back on Begin is the session you actually got. If the service cannot supply word timings for a session, Begin echoes word_boundaries: false even though you asked for true, and word_boundaries_requested: true tells you it was the service’s choice. Read the echo before enabling highlighting.
Don’t build a feature that requires them. Highlighting should degrade to no highlighting, not to a broken player. If your UI cannot work without word timings, this is not the right foundation for it.
// Fragment. message is the parsed frame in your onmessage handler;// disableHighlighting is your UI hook.const disableHighlighting = () => {};if (message.type === "Begin") { const available = message.configuration.word_boundaries === true; if (!available) disableHighlighting();}
In the SDKs
The SDKs open a session with word boundaries off. Turn them on with word_boundaries=True on a Python TTSSession, or wordBoundaries: true on a JavaScript session, and subscribe to the event. Each event carries the frame’s fields: flush_id, audio_start_ms and words, each word with word, start, end and confidence. It arrives after the request’s audio, so keep the audio you played and match the timings to it as above. Neither SDK exposes word boundaries over HTTP; use /v1/speech/events directly for that.
Python SDK
JavaScript SDK
# Fragment. You supply play().import assemblyai as aaiconfig = aai.TTSConfig(voice_id="aai-kyla")def on_words(event): for w in event.words: print(f"request {event.flush_id}: {w.word!r} at {w.start - event.audio_start_ms} ms")with aai.TTSSession(config, word_boundaries=True) as session: session.on(aai.TTSEvents.WordBoundaries, on_words) for chunk in session.speak("Hi, thanks for calling. How can I help you today?"): play(chunk)
// Fragment. You supply play().import { AssemblyAI } from "assemblyai";const client = new AssemblyAI({ apiKey: process.env.ASSEMBLYAI_API_KEY });const session = client.tts.session({ voiceId: "aai-kyla", wordBoundaries: true });session.on("wordBoundaries", (event) => { for (const w of event.words) console.log(`request ${event.flush_id}: ${w.word} at ${w.start - event.audio_start_ms} ms`);});await session.connect();for await (const chunk of session.speak("Hi, thanks for calling. How can I help you today?")) play(chunk);await session.close();
Over the HTTP API
On the HTTP API, POST /v1/speech/events takes the body of Create speech, without the file formats, and answers server-sent events instead of raw audio. They are the frames of the WebSocket API, with the audio in base64 and, after it, the time at which each word is spoken. A stream looks like this, with Begin.configuration and Termination abridged to the fields this section describes:
A successful stream is Begin, one or more Audio, FlushDone, usually one WordBoundaries, then Termination. As on /v1/speech, the status waits for the first audio, and Begin arrives with it.
Audio carries base64 of raw pcm_s16le, pcm_mulaw or pcm_alaw audio, about 80 ms per event. MP3, WAV, bit_rate and "stream": false are refused on this route, because the word times are positions in that raw audio.
WordBoundaries has one entry per whitespace-delimited token of your text, in order, with its punctuation, and start and end in milliseconds from the first sample. It follows the rules above: best effort, with zero-length entries for tokens that make no sound. Termination.word_boundaries_status says whether it was sent: complete, none when the timings could not be computed, or disabled when you sent "word_boundaries": false. Termination.billed_characters is the request’s final billed count.
A failure after the first audio sends one Error event, with the fields of the WebSocket Error frame plus code, message, request_id and status, and the stream ends. A stream that ends with neither Termination nor Error was cut, by the network or by a server failure: send the request again.
While nothing is sent for 15 seconds, a : keepalive comment arrives. Skip lines that start with :, and ignore event types and fields you do not know: Begin.configuration and Termination carry more fields than this section describes.
import base64import jsonimport osimport httpxURL = "https://tts.us.assemblyai.com/v1/speech/events"HEADERS = {"Authorization": os.environ["ASSEMBLYAI_API_KEY"]}TEXT = "Hello there, friend. The weather is lovely today."audio = bytearray()words = []ended = Falsewith httpx.stream( "POST", URL, headers=HEADERS, json={"text": TEXT}, timeout=httpx.Timeout(5.0, read=30.0)) as response: if response.status_code != 200: response.read() raise SystemExit(f"{response.status_code}: {response.text[:300]}") event = None for line in response.iter_lines(): if line.startswith(":"): continue # keepalive if line.startswith("event: "): event = line[len("event: "):] elif line.startswith("data: "): data = json.loads(line[len("data: "):]) if event == "Audio": audio += base64.b64decode(data["audio"]) elif event == "WordBoundaries": words = data["words"] elif event == "Termination": ended = True elif event == "Error": raise SystemExit(f"{data['code']}: {data['message']} ({data['request_id']})")if not ended: raise SystemExit("the stream was cut before its last event")with open("out.pcm", "wb") as f: f.write(audio)for word in words: print(f"{word['start']:>6} ms {word['word']}")
Message frames: every frame, and where WordBoundaries sits in the order.
Was this page helpful?
{"docsConfig":{"theme":"aspen","$schema":"https://mintlify.com/docs.json","name":"AssemblyAI","description":"AssemblyAI provides production-ready Voice AI models for speech-to-text, speaker detection, sentiment analysis, and more. Build with pre-recorded or streaming audio using REST APIs and WebSockets.","colors":{"primary":"#614fd2","light":"#887bdd","dark":"#614fd2"},"logo":{"light":"https://mintcdn.com/assemblyai-fff14a6f/vMuNaI_PbWUbVeV0/assets/assemblyai-lockup-primary.svg?fit=max\u0026auto=format\u0026n=vMuNaI_PbWUbVeV0\u0026q=85\u0026s=808d2a677e2536423045d44b3f330ac2","dark":"https://mintcdn.com/assemblyai-fff14a6f/vMuNaI_PbWUbVeV0/assets/assemblyai-lockup-secondary.svg?fit=max\u0026auto=format\u0026n=vMuNaI_PbWUbVeV0\u0026q=85\u0026s=065ec1618693f8650241242ec62ed0ad","href":"https://www.assemblyai.com"},"favicon":"https://mintcdn.com/assemblyai-fff14a6f/6OQYF4QMRVQRoCb1/assets/favicon.png?fit=max\u0026auto=format\u0026n=6OQYF4QMRVQRoCb1\u0026q=85\u0026s=74b6b6a7321938bd113d4a271e06068d","api":{"openapi":["openapi.yaml","api-reference/specs/llm-gateway.yaml","api-reference/specs/streaming-token.yaml"],"asyncapi":["api-reference/specs/streaming.yaml","api-reference/specs/streaming-tts.yaml"]},"background":{"color":{"light":"#fdfcf8","dark":"#1d1b16"}},"navbar":{"links":[{"href":"https://www.assemblyai.com/playground","label":"Playground"},{"href":"https://www.assemblyai.com/changelog","label":"Changelog"}],"primary":{"type":"button","label":"Sign In","href":"https://www.assemblyai.com/dashboard/login"}},"footer":{"socials":{"x":"https://x.com/AssemblyAI","github":"https://github.com/AssemblyAI","linkedin":"https://www.linkedin.com/company/assemblyai"},"links":[{"header":"Legal","items":[{"label":"Terms of Service","href":"https://www.assemblyai.com/legal/terms-of-service"},{"label":"Privacy Policy","href":"https://www.assemblyai.com/legal/privacy-policy"},{"label":"Trust Center","href":"https://app.vanta.com/assemblyai/trust/7n80syl8zln1bn1qm3x8eg"},{"label":"Subprocessors","href":"https://app.vanta.com/assemblyai/trust/7n80syl8zln1bn1qm3x8eg/subprocessors"}]}]},"seo":{"metatags":{"canonical":"https://www.assemblyai.com/docs"}},"fonts":{"heading":{"family":"GT Standard","weight":600,"source":"https://www.assemblyai.com/_aai/fonts/rebrand/GT-Standard-M-Semibold.woff2","format":"woff2"},"body":{"family":"GT Standard","weight":400,"source":"https://www.assemblyai.com/_aai/fonts/rebrand/GT-Standard-M-Regular.woff2","format":"woff2"}},"icons":{"library":"fontawesome"},"banner":{"content":"**New** AssemblyAI-TTS-1: Text-to-speech for voice agents. [Learn more](https://www.assemblyai.com/docs/tts/getting-started/quickstart)","dismissible":true},"contextual":{"options":["copy","view","chatgpt","claude","download-spec"],"display":"header"},"styling":{"latex":true},"navigation":{"pages":[]}},"deployment":{"deploymentMetadataValue":{"subdomain":"assemblyai-fff14a6f","hasAudioNarration":false,"actualSubdomain":"assemblyai-fff14a6f","customDomains":["assemblyai.com"],"gitSource":{"type":"github","owner":"assemblyai","repo":"assemblyai-docs","deployBranch":"master","contentDirectory":"","isPrivate":true,"mountPath":"","label":""},"feedback":{"thumbs":true,"agentFeedback":true},"entitlements":{"AI_CHAT":{"status":"ENABLED"},"REMOVE_BRANDING":{"status":"ENABLED"},"CONTEXTUAL_FEEDBACK":{"status":"ENABLED"},"CODE_SNIPPET_FEEDBACK":{"status":"ENABLED"},"WEB_MCP":{"status":"ENABLED"},"NARRATION":{"status":"ENABLED"}},"buildId":"6afc634e8fbc458f58e3802e","clientVersion":"0.0.4167","searchFilterCounts":[{"language":"default","tag":"Overview","count":116},{"language":"default","tag":"FAQ","count":94},{"language":"default","tag":"Pre-recorded STT","count":71},{"language":"default","tag":"Guides","count":59},{"language":"default","tag":"Real-time STT","count":50},{"language":"default","tag":"Tutorials","count":48},{"language":"default","tag":"Pre-recorded audio","count":47},{"language":"default","tag":"LLM Gateway","count":45},{"language":"default","tag":"Features","count":39},{"language":"default","tag":"API reference","count":35}]},"assistantConfig":{"enableAskAiButton":false,"enableStarterQuestions":false,"enableContextualStarterQuestions":false,"enableAskAiSearchBar":true,"attachments":{"status":"DISABLED"}},"basePath":"/docs","hasChatPermissions":true,"hasWebMcpPermissions":true,"toggles":[{"name":"kill-isr","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-rbac-roles","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-mintlify-slack","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-integrations-settings","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"rbac-expansion","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-pr-grouping","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-prefix","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"analytics-v7","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"user-flows-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"no-publish-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-onboarding","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving-live","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"stripTemperatureMiddleware","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"support-platform-pylon-zendesk","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"instant-rollback","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications-server","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-fable","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fixed-ai-action-pricing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automation-resolve-merge-conflict","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"scoped-publishing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"mintql","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"page-level-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-version-history-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"billing-page-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"full-page-search","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workspace-projects","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"assistant-v5","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fork-pr-previews","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"revalidation-jwt","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-sso-scim","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"hobby-abuse-scan","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"dashboard-inbox","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-signup","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-dual-write","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-enabled","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"approve-review-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-metadata","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"sso-setup-deadline-banner","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-serve","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"live-agents","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"starter-pricing","enabled":true,"variant":{"name":"treatment","enabled":true,"payload":{"type":"string","value":"treatment"},"feature_enabled":true},"impressionData":false},{"name":"llms-txt-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false}],"googleFontsInlined":false},"customCss":[{"filePath":"style.css","content":"@font-face{font-family:\"GT Standard\";src:url(\"https://www.assemblyai.com/_aai/fonts/rebrand/GT-Standard-M-Regular.woff2\")\n format(\"woff2\"),\n url(\"https://www.assemblyai.com/docs/assets/GT-Standard-M-Regular.woff2\")\n format(\"woff2\"),\n url(\"/assets/GT-Standard-M-Regular.woff2\") format(\"woff2\");font-weight:400;font-style:normal;font-display:swap}@font-face{font-family:\"GT Standard\";src:url(\"https://www.assemblyai.com/_aai/fonts/rebrand/GT-Standard-M-Medium.woff2\")\n format(\"woff2\"),\n url(\"https://www.assemblyai.com/docs/assets/GT-Standard-M-Medium.woff2\")\n format(\"woff2\"),\n url(\"/assets/GT-Standard-M-Medium.woff2\") format(\"woff2\");font-weight:500;font-style:normal;font-display:swap}@font-face{font-family:\"GT Standard\";src:url(\"https://www.assemblyai.com/_aai/fonts/rebrand/GT-Standard-M-Semibold.woff2\")\n format(\"woff2\"),\n url(\"https://www.assemblyai.com/docs/assets/GT-Standard-M-Semibold.woff2\")\n format(\"woff2\"),\n url(\"/assets/GT-Standard-M-Semibold.woff2\") format(\"woff2\");font-weight:600;font-style:normal;font-display:swap}@font-face{font-family:\"Modern Gothic Mono\";src:url(\"https://www.assemblyai.com/_aai/fonts/rebrand/ModernGothicMono-Light.woff2\")\n format(\"woff2\");font-weight:300;font-style:normal;font-display:swap}@font-face{font-family:\"Modern Gothic Mono\";src:url(\"https://www.assemblyai.com/_aai/fonts/rebrand/ModernGothicMono-Regular.woff2\")\n format(\"woff2\");font-weight:400;font-style:normal;font-display:swap}@font-face{font-family:\"Modern Gothic Mono\";src:url(\"https://www.assemblyai.com/_aai/fonts/rebrand/ModernGothicMono-Medium.woff2\")\n format(\"woff2\");font-weight:500;font-style:normal;font-display:swap}@font-face{font-family:\"Modern Gothic Mono\";src:url(\"https://www.assemblyai.com/_aai/fonts/rebrand/ModernGothicMono-Italic.woff2\")\n format(\"woff2\");font-weight:400;font-style:italic;font-display:swap}html:root{--font-family-body-custom: \"GT Standard\", -apple-system, BlinkMacSystemFont,\n \"Segoe UI\", sans-serif;--font-family-headings-custom: \"GT Standard\", -apple-system, BlinkMacSystemFont,\n \"Segoe UI\", sans-serif}code,kbd,samp,pre,pre code,pre code span{font-family:\"Modern Gothic Mono\", \"JetBrains Mono\", ui-monospace, SFMono-Regular,\n Menlo, Consolas, monospace!important}html{overflow-y:scroll;scrollbar-gutter:stable}pre,pre code,pre code span{font-size:0.75rem!important;line-height:1.5!important}:not(pre)>code{font-size:0.75rem!important;line-height:inherit!important}[data-nav-tag=\"WSS\"]{color:#614fd2!important;background-color:rgba(136, 123, 221, 0.14)!important;font-size:0.55rem!important}:is(.dark) [data-nav-tag=\"WSS\"]{color:#887bdd!important}div:has(>.nav-tag-pill>[data-nav-tag=\"WSS\"]){order:-1}.nav-tag-pill:has(>[data-nav-tag=\"WSS\"]){width:2.375rem;justify-content:flex-start}.ws-endpoint{display:flex;align-items:center;gap:0.625rem;margin:1rem 0 1.75rem;padding:0.5rem 0.75rem;border:1px solid rgba(0, 0, 0, 0.08);border-radius:0.5rem;background:rgba(0, 0, 0, 0.02);font-size:0.875rem;overflow-x:auto}:is(.dark) .ws-endpoint{border-color:rgba(255, 255, 255, 0.1);background:rgba(255, 255, 255, 0.03)}.ws-endpoint-badge{flex:none;font-weight:700;font-size:0.75rem;letter-spacing:0.02em;color:#614fd2;background:rgba(136, 123, 221, 0.14);padding:0.1875rem 0.5rem;border-radius:0.375rem}:is(.dark) .ws-endpoint-badge{color:#887bdd}.ws-endpoint-url{font-family:ui-monospace, SFMono-Regular, \"SF Mono\", Menlo, Consolas, monospace;white-space:nowrap}#assistant-entry,:is(.dark) #assistant-entry{background-color:#614fd2!important;filter:none!important}#assistant-entry:hover,:is(.dark) #assistant-entry:hover{background-color:#614fd2!important;opacity:0.9;filter:none!important}#assistant-entry span,#assistant-entry svg,:is(.dark) #assistant-entry span,:is(.dark) #assistant-entry svg{color:#ffffff!important}.callout strong{font-weight:700}*{-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale;-o-font-smoothing:antialiased}.katex-display{overflow-x:visible!important;overflow-y:visible!important}.katex-display>.katex{display:flex!important;justify-content:center;width:100%!important;overflow-x:visible!important;overflow-y:visible!important}.katex-display>.katex>.katex-html{display:inline-block!important;width:auto!important;max-width:100%;overflow-x:visible!important;overflow-y:visible!important}.card-logo-icon{display:block;width:1.5rem!important;height:1.5rem!important;min-width:1.5rem;max-width:1.5rem;max-height:1.5rem;margin:0!important;object-fit:contain;object-position:center;vertical-align:middle}.card-logo-icon-livekit{transform:scale(0.9)}.card-logo-icon-pipecat{transform:scale(1.12)}.card-logo-icon-livekit,.card-logo-icon-pipecat,.card-logo-icon-vercel{color:#614fd2}:is(.dark) .card-logo-icon-livekit,:is(.dark) .card-logo-icon-pipecat,:is(.dark) .card-logo-icon-vercel{color:#887bdd}.recommended-pill{background:#614fd2;color:#fff;padding:2px 8px;border-radius:9999px;font-size:0.7rem;font-weight:600;margin-left:6px;vertical-align:middle;white-space:nowrap}:is(.dark) .recommended-pill{background:#614fd2}.preview-pill{background:#b45309;color:#fff;padding:2px 8px;border-radius:9999px;font-size:0.7rem;font-weight:600;margin-left:6px;vertical-align:middle;white-space:nowrap}:is(.dark) .preview-pill{background:#b45309}table code{white-space:nowrap}"}],"page":{"pagePath":"tts/word-boundaries","slug":"/tts/word-boundaries","pageMetadata":{"title":"Word boundaries","description":"Per-word timings for synthesized speech, for highlighting text in sync with playback.","href":"/tts/word-boundaries"},"description":{"compiledSource":"\"use strict\";\nconst {jsx: _jsx} = arguments[0];\nconst {useMDXComponents: _provideComponents} = arguments[0];\nfunction _createMdxContent(props) {\n const _components = {\n p: \"p\",\n ..._provideComponents(),\n ...props.components\n };\n return _jsx(_components.p, {\n children: \"Per-word timings for synthesized speech, for highlighting text in sync with playback.\"\n });\n}\nfunction MDXContent(props = {}) {\n const {wrapper: MDXLayout} = {\n ..._provideComponents(),\n ...props.components\n };\n return MDXLayout ? _jsx(MDXLayout, {\n ...props,\n children: _jsx(_createMdxContent, {\n ...props\n })\n }) : _createMdxContent(props);\n}\nreturn {\n default: MDXContent\n};\n","frontmatter":{},"scope":{}},"pageType":"default","mdx":{"mdxSource":{"compiledSource":"\"use strict\";const{Fragment:_Fragment,jsx:_jsx,jsxs:_jsxs}=arguments[0];const{useMDXComponents:_provideComponents}=arguments[0];function _createMdxContent(props){const _components={a:`a`,code:`code`,li:`li`,p:`p`,pre:`pre`,span:`span`,strong:`strong`,ul:`ul`,..._provideComponents(),...props.components},{CodeBlock,Heading,Tab,Tabs,Warning}=_components;if(!CodeBlock)_missingMdxReference(`CodeBlock`,true);if(!Heading)_missingMdxReference(`Heading`,true);if(!Tab)_missingMdxReference(`Tab`,true);if(!Tabs)_missingMdxReference(`Tabs`,true);if(!Warning)_missingMdxReference(`Warning`,true);return _jsxs(_Fragment,{children:[_jsx(_components.p,{children:`Word boundaries tell you when each word is spoken in the audio being streamed to you. The usual reason to want them is karaoke-style highlighting: following along in the source text while the speech plays.`}),`\n`,_jsxs(_components.p,{children:[`They are on by default on the wire. Pass `,_jsx(_components.code,{children:`word_boundaries=false`}),` at connect time to turn them off. The SDKs turn them off unless you ask for them; see `,_jsx(_components.a,{href:`#in-the-sdks`,children:`In the SDKs`}),`. On the HTTP API they come from a route of their own; see `,_jsx(_components.a,{href:`#over-the-http-api`,children:`Over the HTTP API`}),`.`]}),`\n`,_jsx(Heading,{level:`2`,id:`the-frame`,children:`The frame`}),`\n`,_jsxs(_components.p,{children:[`You receive at most one `,_jsx(_components.code,{children:`WordBoundaries`}),` frame per request (each `,_jsx(_components.code,{children:`Flush`}),` concludes one request), and it covers the spoken words in that request’s text, however long the request is. This one is for a request whose text began `,_jsx(_components.code,{children:`Hello there, friend,`}),`:`]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`10`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`10`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"WordBoundaries\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"flush_id\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"audio_start_ms\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"words\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: [`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` { `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"word\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Hello\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"start\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`265`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"end\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`530`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"confidence\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`0.4791773557662964`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` },`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` { `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"word\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"there,\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"start\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`546`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"end\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`712`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"confidence\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`0.9253882169723511`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` },`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` { `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"word\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"friend,\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"start\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`845`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"end\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`1126`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"confidence\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`0.8478114008903503`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` }`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` ]`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})}),`\n`,_jsxs(_components.ul,{children:[`\n`,_jsxs(_components.li,{children:[_jsx(_components.code,{children:`flush_id`}),` is the request the words belong to. It is the same id that the request’s `,_jsx(_components.code,{children:`FlushDone`}),` carries.`]}),`\n`,_jsxs(_components.li,{children:[_jsx(_components.code,{children:`audio_start_ms`}),` is where this request’s audio begins on the session’s audio timeline (see below). It is `,_jsx(_components.code,{children:`0`}),` for the first request.`]}),`\n`,_jsxs(_components.li,{children:[_jsx(_components.code,{children:`words`}),` has one entry per whitespace-delimited token of the request’s text, in the order you sent them. Each entry has four keys:`,`\n`,_jsxs(_components.ul,{children:[`\n`,_jsxs(_components.li,{children:[_jsx(_components.code,{children:`word`}),`: the word as it appeared in your text.`]}),`\n`,_jsxs(_components.li,{children:[_jsx(_components.code,{children:`start`}),` and `,_jsx(_components.code,{children:`end`}),`: when the word is spoken, in milliseconds.`]}),`\n`,_jsxs(_components.li,{children:[_jsx(_components.code,{children:`confidence`}),`: a number from `,_jsx(_components.code,{children:`0`}),` to `,_jsx(_components.code,{children:`1`}),` for that word’s timing.`]}),`\n`]}),`\n`]}),`\n`]}),`\n`,_jsxs(_components.p,{children:[`All times are integer milliseconds and do not depend on the session’s `,_jsx(_components.code,{children:`sample_rate`}),` or `,_jsx(_components.code,{children:`encoding`}),`.`]}),`\n`,_jsxs(Heading,{level:`3`,id:`what-word-contains`,children:[`What `,_jsx(_components.code,{children:`word`}),` contains`]}),`\n`,_jsxs(_components.p,{children:[_jsx(_components.code,{children:`word`}),` is a whitespace-delimited token from your text, exactly as you sent it, with its punctuation still attached: `,_jsx(_components.code,{children:`there,`}),`, `,_jsx(_components.code,{children:`today?`}),`, `,_jsx(_components.code,{children:`dog.`}),`. The entries are one to one with the tokens you get by splitting the request’s text on whitespace, in the same order, so the `,_jsx(_components.code,{children:`n`}),`th entry is the `,_jsx(_components.code,{children:`n`}),`th token. Non-ASCII text is returned as is.`]}),`\n`,_jsxs(_components.p,{children:[`A token with no sound of its own, such as the dash in `,_jsx(_components.code,{children:`Hello - world`}),` or the `,_jsx(_components.code,{children:`?`}),` in the French `,_jsx(_components.code,{children:`ça va ?`}),`, still gets an entry. It has `,_jsx(_components.code,{children:`start`}),` equal to `,_jsx(_components.code,{children:`end`}),` and a `,_jsx(_components.code,{children:`confidence`}),` of `,_jsx(_components.code,{children:`0`}),`, and sits at the edge of the word next to it: an opening mark such as `,_jsx(_components.code,{children:`¿`}),` or `,_jsx(_components.code,{children:`(`}),` at the start of the word after it, any other token at the end of the word before it. `,_jsx(_components.code,{children:`Bonjour, ça va ?`}),` gives four entries, the last one `,_jsx(_components.code,{children:`?`}),` at the end of `,_jsx(_components.code,{children:`va`}),`. A token in square brackets, such as `,_jsx(_components.code,{children:`[laughs]`}),` or `,_jsx(_components.code,{children:`[pause]`}),`, also gets a zero-length entry, placed the same way. Such a tag is not an instruction: the model reads the word inside it aloud, yet its entry stays zero-length (see `,_jsx(_components.a,{href:`/tts/prompting-guide#how-text-is-read`,children:`How text is read`}),`). Skip zero-length entries when you highlight.`]}),`\n`,_jsxs(_components.p,{children:[`A token the model reads as several words, such as `,_jsx(_components.code,{children:`$1,234,567`}),`, gets one entry that spans all of them, several seconds long in that case. Markup stays in `,_jsx(_components.code,{children:`word`}),` as you sent it: `,_jsx(_components.code,{children:`**Hello**`}),` is one entry, timed over the spoken `,_jsx(_components.code,{children:`Hello`}),`, since markup is not read out. An emoji standing alone between spaces gets a zero-length entry, since emoji are not read out either. Timing follows your text, not the spoken form, so numbers, symbols, URLs and tags can get stretched or zero-length spans with a low `,_jsx(_components.code,{children:`confidence`}),`. See `,_jsx(_components.a,{href:`/tts/prompting-guide#how-text-is-read`,children:`How text is read`}),`.`]}),`\n`,_jsx(_components.p,{children:`A frame never lists words the audio does not contain. When the audio of a request leaves part of its text unspoken, the request gets no frame at all.`}),`\n`,_jsx(Heading,{level:`2`,id:`times-are-on-the-session-timeline`,children:`Times are on the session timeline`}),`\n`,_jsxs(_components.p,{children:[_jsx(_components.code,{children:`start`}),` and `,_jsx(_components.code,{children:`end`}),` are positions on the session’s continuous output-audio timeline: the audio delivered since the session began, across every request. They are not relative to the request’s own audio. If the first request produced 2800 ms of audio, the second request’s frame has `,_jsx(_components.code,{children:`audio_start_ms: 2800`}),` and its first word starts at 2800 or later.`]}),`\n`,_jsx(_components.p,{children:`Two consequences:`}),`\n`,_jsxs(_components.ul,{children:[`\n`,_jsxs(_components.li,{children:[_jsxs(_components.strong,{children:[`Do not add `,_jsx(_components.code,{children:`audio_start_ms`}),` to `,_jsx(_components.code,{children:`start`}),`.`]}),` `,_jsx(_components.code,{children:`start`}),` is already a position in the concatenated session audio. Adding the offset misplaces every highlight after the first request.`]}),`\n`,_jsxs(_components.li,{children:[_jsx(_components.strong,{children:`Subtract it to get a position inside the request’s own audio.`}),` `,_jsx(_components.code,{children:`start - audio_start_ms`}),` is where the word falls in that request’s `,_jsx(_components.code,{children:`Audio`}),` frames.`]}),`\n`]}),`\n`,_jsxs(_components.p,{children:[`The timeline never rewinds. Every byte of audio delivered to you holds its place on it, including the audio of a request you then cancelled and the short tail that arrives between your `,_jsx(_components.code,{children:`Cancel`}),` and the server’s `,_jsx(_components.code,{children:`Cancelled`}),`, so the next request’s `,_jsx(_components.code,{children:`audio_start_ms`}),` counts audio you may have dropped and never played. That is why the mapping to your player’s clock below anchors on each request rather than on the session.`]}),`\n`,_jsx(Heading,{level:`2`,id:`when-it-arrives`,children:`When it arrives`}),`\n`,_jsxs(_components.p,{children:[`Each `,_jsx(_components.code,{children:`WordBoundaries`}),` frame is sent after the `,_jsx(_components.code,{children:`FlushDone`}),` for the same `,_jsx(_components.code,{children:`flush_id`}),`, usually within tens of milliseconds. When you have several requests in flight it often lands after the next request’s first `,_jsx(_components.code,{children:`Audio`}),` frames have already arrived. Match frames to requests by `,_jsx(_components.code,{children:`flush_id`}),`, never by arrival order. For example, a sequence with three pipelined requests:`]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`3`,language:`text`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`text`,children:_jsxs(_components.code,{language:`text`,numberOfLines:`3`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`Begin | Audio(0) x43 | FlushDone(0) | Audio(1) x2 | WordBoundaries(0) | Audio(1) x37 | FlushDone(1)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`| Audio(2) x3 | WordBoundaries(1) | Audio(2) x42 | FlushDone(2) | WordBoundaries(2) | Termination | close 1000`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})})]})})}),`\n`,_jsxs(_components.p,{children:[`Nothing arrives after `,_jsx(_components.code,{children:`Termination`}),`. `,_jsx(_components.code,{children:`Terminate`}),` waits for pending word boundaries before it ends the session, so you can send it right after your last `,_jsx(_components.code,{children:`Flush`}),` and still receive that request’s frame. Text you never flushed is flushed for you by `,_jsx(_components.code,{children:`Terminate`}),` and gets a frame of its own. The same drain runs when the session reaches the `,_jsx(_components.code,{children:`expires_at`}),` time announced in `,_jsx(_components.code,{children:`Begin`}),`.`]}),`\n`,_jsxs(_components.p,{children:[_jsx(_components.code,{children:`FlushDone`}),` means the request’s audio has been delivered to the socket, not that it has finished playing. Synthesis runs several times faster than real time, so when the `,_jsx(_components.code,{children:`WordBoundaries`}),` frame arrives the whole request is usually buffered on your side and its first words may already be playing. Schedule highlights against your audio clock instead of reacting to each frame as it arrives, and apply a highlight immediately when its time has already passed.`]}),`\n`,_jsx(_components.p,{children:`Word boundaries do not delay the audio: a request’s first audio arrives as soon with them as without, so turning them off does not make it faster. A request’s frame comes only after all of its audio, though, so a long reply sent as one request gets its timings once its whole audio has been sent. To highlight from the first sentence on, flush each sentence as a request of its own: its timings then arrive right after that sentence’s audio. Each request is spoken on its own, so this adds a short pause between sentences.`}),`\n`,_jsxs(_components.p,{children:[`You can print the frames yourself. The program below sends two requests and prints each `,_jsx(_components.code,{children:`FlushDone`}),` and `,_jsx(_components.code,{children:`WordBoundaries`}),` as it arrives, with every word’s position on the session timeline and inside its own request. Set `,_jsx(_components.code,{children:`ASSEMBLYAI_API_KEY`}),` and run it with the `,_jsx(_components.code,{children:`websockets`}),` package (14 or newer):`]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`41`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`41`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` asyncio`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` json`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` os`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` websockets `}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# pip install -U websockets`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#D4D4D4`},children:`URL`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` =`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"wss://tts.us.assemblyai.com/v1/ws?voice_id=aai-kyla\u0026language=english\"`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#D4D4D4`},children:`API_KEY`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` =`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` os.environ[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"ASSEMBLYAI_API_KEY\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#D4D4D4`},children:`REQUESTS`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` =`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` [`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Hello there, friend.\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Pack my box with five dozen liquor jugs.\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`async`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` def`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#DCDCAA`},children:` main`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`():`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` async`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` with`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` websockets.connect(`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#D4D4D4`},children:`URL`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#9CDCFE`},children:`additional_headers`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Authorization\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#D4D4D4`},children:`API_KEY`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}) `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`as`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` ws:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` first `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` json.loads(`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`await`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` ws.recv())`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` if`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` first[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`] `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`==`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"Error\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` raise`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#4EC9B0`},children:` SystemExit`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`first[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`'error_code'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`: `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`first[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`'error'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`)`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` if`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` first[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"configuration\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`][`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"word_boundaries\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`] `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`is`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` not`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:` True`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` raise`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#4EC9B0`},children:` SystemExit`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"this session will not send WordBoundaries\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`)`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` for`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` text `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`in`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#D4D4D4`},children:` REQUESTS`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` await`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` ws.send(json.dumps({`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Generate\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"text\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: text}))`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` await`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` ws.send(json.dumps({`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Flush\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}))`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` await`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` ws.send(json.dumps({`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Terminate\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}))`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` async`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` for`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` raw `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`in`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` ws:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` message `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` json.loads(raw)`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` if`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` message[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`] `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`==`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"FlushDone\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#DCDCAA`},children:` print`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"FlushDone flush_id=`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`message[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`'flush_id'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` audio_duration_ms=`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`message[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`'audio_duration_ms'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`)`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` elif`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` message[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`] `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`==`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"WordBoundaries\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#DCDCAA`},children:` print`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"WordBoundaries flush_id=`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`message[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`'flush_id'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` audio_start_ms=`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`message[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`'audio_start_ms'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`)`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` for`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` word `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`in`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` message[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"words\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` in_request `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` word[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"start\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`] `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`-`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` message[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"audio_start_ms\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#DCDCAA`},children:` print`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\" `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`word[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`'word'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`:\u003c10}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` session `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`word[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`'start'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`:>5}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` to `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`word[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`'end'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`:>5}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` ms request +`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`in_request`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` ms\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`)`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` elif`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` message[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`] `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`==`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"Termination\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` break`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` elif`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` message[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`] `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`==`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"Error\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` raise`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#4EC9B0`},children:` SystemExit`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`message[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`'error_code'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`: `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`message[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`'error'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`)`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`asyncio.run(main())`})}),`\n`]})})}),`\n`,_jsx(Heading,{level:`2`,id:`mapping-timings-to-playback`,children:`Mapping timings to playback`}),`\n`,_jsxs(_components.p,{children:[`A word time is a position in the audio, and your player has its own clock. To turn one into the other, remember the player time at which a request’s first audio buffer was scheduled and offset from there. In a WebAudio client that is the `,_jsx(_components.code,{children:`AudioContext`}),` time you passed to `,_jsx(_components.code,{children:`source.start()`}),` for the request’s first `,_jsx(_components.code,{children:`Audio`}),` frame:`]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`17`,language:`javascript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`javascript`,children:_jsxs(_components.code,{language:`javascript`,numberOfLines:`17`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`// Fragment: the scheduling side of a WebAudio client.`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`const`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#4FC1FF`},children:` ctx`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` =`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` new`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#DCDCAA`},children:` AudioContext`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(); `}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`// plays the session's audio`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`const`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#4FC1FF`},children:` anchors`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` =`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` new`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#DCDCAA`},children:` Map`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(); `}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`// flush_id -> AudioContext time at which that request's`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:` // first Audio buffer was scheduled; set it where you enqueue audio`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`const`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#DCDCAA`},children:` highlightWord`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` =`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` (`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#9CDCFE`},children:`word`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`) `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`=>`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:` console`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#DCDCAA`},children:`log`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"speaking:\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`word`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`); `}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`// your UI hook`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`function`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#DCDCAA`},children:` highlight`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#9CDCFE`},children:`message`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`) {`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` const`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#4FC1FF`},children:` anchor`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` =`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:` anchors`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#DCDCAA`},children:`get`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`message`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`flush_id`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`);`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` if`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` (`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`anchor`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` ===`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:` undefined`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`) `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`return`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`; `}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`// no audio was scheduled for this request`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` for`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` (`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`const`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#4FC1FF`},children:` word`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` of`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:` message`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`words`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`) {`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` if`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` (`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`word`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`start`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` ===`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:` word`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`end`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`) `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`continue`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`; `}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`// nothing is spoken for this token`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:` // start is on the session timeline; the anchor is at audio_start_ms on that timeline.`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` const`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#4FC1FF`},children:` at`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` =`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:` anchor`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` +`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` (`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`word`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`start`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` -`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:` message`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`audio_start_ms`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`) `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`/`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:` 1000`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`;`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` const`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#4FC1FF`},children:` delay`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` =`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:` Math`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#DCDCAA`},children:`max`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, (`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`at`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` -`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:` ctx`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`currentTime`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`) `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`*`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:` 1000`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`);`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#DCDCAA`},children:` setTimeout`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(() `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`=>`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#DCDCAA`},children:` highlightWord`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`word`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`word`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`), `}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`delay`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`);`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` }`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})}),`\n`,_jsxs(_components.p,{children:[`Anchoring on each request is what keeps highlights correct after a `,_jsx(_components.code,{children:`Cancel`}),`, when the audio you stopped playing still counts on the server’s timeline. If you play everything the server sends as one uninterrupted stream and never cancel, a single anchor at the session’s first sample is enough: highlight at `,_jsx(_components.code,{children:`sessionStart + word.start / 1000`}),`.`]}),`\n`,_jsx(_components.p,{children:`Clear pending highlight timers together with your scheduled audio when you cancel or start a new request.`}),`\n`,_jsx(Heading,{level:`2`,id:`treat-them-as-best-effort`,children:`Treat them as best-effort`}),`\n`,_jsx(Warning,{children:_jsxs(_components.p,{children:[_jsx(_components.code,{children:`WordBoundaries`}),` frames are best-effort. A request can complete, with its\naudio delivered in full, and never get one. Nothing is sent in the frame’s\nplace, and the audio stream itself is unaffected either way.`]})}),`\n`,_jsx(_components.p,{children:`You will not get a frame for:`}),`\n`,_jsxs(_components.ul,{children:[`\n`,_jsxs(_components.li,{children:[`a request you cancelled with `,_jsx(_components.code,{children:`Cancel`}),`;`]}),`\n`,_jsxs(_components.li,{children:[`a request containing a stretch the server speaks in one piece whose audio runs past about 32 seconds. The server splits text into such pieces at sentence ends, and inside a sentence that runs long (see `,_jsx(_components.a,{href:`/tts/prompting-guide#how-text-is-read`,children:`How text is read`}),`), so this is rare. Other requests in the same session still get theirs;`]}),`\n`,_jsxs(_components.li,{children:[`a `,_jsx(_components.code,{children:`Flush`}),` with nothing to speak, which gets a `,_jsx(_components.code,{children:`FlushDone`}),` with `,_jsx(_components.code,{children:`audio_duration_ms: 0`}),` and no `,_jsx(_components.code,{children:`WordBoundaries`}),`;`]}),`\n`,_jsx(_components.li,{children:`a request whose audio does not cover all of its text;`}),`\n`,_jsx(_components.li,{children:`a request for which the server could not compute timings, for example after a transient problem during that request; the audio itself is unaffected.`}),`\n`]}),`\n`,_jsxs(_components.p,{children:[`A long request needs no splitting for highlighting: the server times it in parts of whole sentences and sends one frame that lists every token of the request, after its `,_jsx(_components.code,{children:`FlushDone`}),`.`]}),`\n`,_jsx(_components.p,{children:`Two consequences worth designing around:`}),`\n`,_jsxs(_components.ul,{children:[`\n`,_jsxs(_components.li,{children:[_jsxs(_components.strong,{children:[`Check the `,_jsx(_components.code,{children:`Begin`}),` frame, not your request.`]}),` The `,_jsx(_components.code,{children:`configuration`}),` echoed back on `,_jsx(_components.code,{children:`Begin`}),` is the session you actually got. If the service cannot supply word timings for a session, `,_jsx(_components.code,{children:`Begin`}),` echoes `,_jsx(_components.code,{children:`word_boundaries: false`}),` even though you asked for `,_jsx(_components.code,{children:`true`}),`, and `,_jsx(_components.code,{children:`word_boundaries_requested: true`}),` tells you it was the service’s choice. Read the echo before enabling highlighting.`]}),`\n`,_jsxs(_components.li,{children:[_jsx(_components.strong,{children:`Don’t build a feature that requires them.`}),` Highlighting should degrade to no highlighting, not to a broken player. If your UI cannot work without word timings, this is not the right foundation for it.`]}),`\n`]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`8`,language:`javascript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`javascript`,children:_jsxs(_components.code,{language:`javascript`,numberOfLines:`8`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`// Fragment. message is the parsed frame in your onmessage handler;`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`// disableHighlighting is your UI hook.`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`const`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#DCDCAA`},children:` disableHighlighting`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` =`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` () `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`=>`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` {};`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`if`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` (`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`message`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`type`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` ===`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"Begin\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`) {`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` const`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#4FC1FF`},children:` available`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` =`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:` message`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`configuration`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`word_boundaries`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` ===`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:` true`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`;`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` if`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` (`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`!`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`available`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`) `}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#DCDCAA`},children:`disableHighlighting`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`();`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})}),`\n`,_jsx(Heading,{level:`2`,id:`in-the-sdks`,children:`In the SDKs`}),`\n`,_jsxs(_components.p,{children:[`The SDKs open a session with word boundaries off. Turn them on with `,_jsx(_components.code,{children:`word_boundaries=True`}),` on a Python `,_jsx(_components.code,{children:`TTSSession`}),`, or `,_jsx(_components.code,{children:`wordBoundaries: true`}),` on a JavaScript session, and subscribe to the event. Each event carries the frame’s fields: `,_jsx(_components.code,{children:`flush_id`}),`, `,_jsx(_components.code,{children:`audio_start_ms`}),` and `,_jsx(_components.code,{children:`words`}),`, each word with `,_jsx(_components.code,{children:`word`}),`, `,_jsx(_components.code,{children:`start`}),`, `,_jsx(_components.code,{children:`end`}),` and `,_jsx(_components.code,{children:`confidence`}),`. It arrives after the request’s audio, so keep the audio you played and match the timings to it as above. Neither SDK exposes word boundaries over HTTP; use `,_jsx(_components.code,{children:`/v1/speech/events`}),` directly for that.`]}),`\n`,_jsxs(Tabs,{groupId:`language`,children:[_jsx(Tab,{language:`python-sdk`,title:`Python SDK`,default:true,id:`python-sdk`,children:_jsx(CodeBlock,{filename:``,numberOfLines:`15`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`15`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# Fragment. You supply play().`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` assemblyai `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`as`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` aai`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`config `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` aai.TTSConfig(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#9CDCFE`},children:`voice_id`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"aai-kyla\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`)`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`def`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#DCDCAA`},children:` on_words`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`event`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`):`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` for`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` w `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`in`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` event.words:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#DCDCAA`},children:` print`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"request `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`event.flush_id`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`: `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`w.word`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`!r}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` at `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`w.start `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`-`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` event.audio_start_ms`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` ms\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`)`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`with`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` aai.TTSSession(config, `}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#9CDCFE`},children:`word_boundaries`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:`True`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`) `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`as`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` session:`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` session.on(aai.TTSEvents.WordBoundaries, on_words)`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` for`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` chunk `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`in`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` session.speak(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Hi, thanks for calling. How can I help you today?\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`):`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` play(chunk)`})}),`\n`]})})})}),_jsx(Tab,{language:`javascript-sdk`,title:`JavaScript SDK`,id:`javascript-sdk`,children:_jsx(CodeBlock,{filename:``,numberOfLines:`12`,language:`javascript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`javascript`,children:_jsxs(_components.code,{language:`javascript`,numberOfLines:`12`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`// Fragment. You supply play().`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` { `}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`AssemblyAI`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` } `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`from`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"assemblyai\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`;`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`const`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#4FC1FF`},children:` client`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` =`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` new`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#DCDCAA`},children:` AssemblyAI`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`({ `}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`apiKey:`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:` process`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`env`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#4FC1FF`},children:`ASSEMBLYAI_API_KEY`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` });`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`const`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#4FC1FF`},children:` session`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` =`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:` client`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`tts`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#DCDCAA`},children:`session`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`({ `}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`voiceId:`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"aai-kyla\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`wordBoundaries:`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:` true`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` });`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`session`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#DCDCAA`},children:`on`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"wordBoundaries\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, (`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#9CDCFE`},children:`event`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`) `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`=>`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` {`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` for`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` (`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`const`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#4FC1FF`},children:` w`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` of`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:` event`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`words`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`) `}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`console`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#DCDCAA`},children:`log`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:\"`request \"}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#569CD6`},children:\"${\"}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`event`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`flush_id`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#569CD6`},children:\"${\"}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`w`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`word`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` at `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#569CD6`},children:\"${\"}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`w`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`start`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` -`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:` event`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`audio_start_ms`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:\" ms`\"}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`);`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`});`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`await`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:` session`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#DCDCAA`},children:`connect`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`();`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`for`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` await`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` (`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`const`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#4FC1FF`},children:` chunk`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` of`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:` session`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#DCDCAA`},children:`speak`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Hi, thanks for calling. How can I help you today?\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`)) `}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#DCDCAA`},children:`play`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:`chunk`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`);`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`await`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#9CDCFE`},children:` session`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`.`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#DCDCAA`},children:`close`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`();`})]}),`\n`]})})})})]}),`\n`,_jsx(Heading,{level:`2`,id:`over-the-http-api`,children:`Over the HTTP API`}),`\n`,_jsxs(_components.p,{children:[`On the HTTP API, `,_jsx(_components.a,{href:`/tts/api-reference/http-api/stream-speech-events`,children:_jsx(_components.code,{children:`POST /v1/speech/events`})}),` takes the body of `,_jsx(_components.a,{href:`/tts/api-reference/http-api/create-speech`,children:`Create speech`}),`, without the file formats, and answers `,_jsx(_components.a,{href:`https://html.spec.whatwg.org/multipage/server-sent-events.html`,children:`server-sent events`}),` instead of raw audio. They are the frames of the WebSocket API, with the audio in base64 and, after it, the time at which each word is spoken. A stream looks like this, with `,_jsx(_components.code,{children:`Begin.configuration`}),` and `,_jsx(_components.code,{children:`Termination`}),` abridged to the fields this section describes:`]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`15`,language:`text`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`text`,children:_jsxs(_components.code,{language:`text`,numberOfLines:`15`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`event: Begin`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`data: {\"type\":\"Begin\",\"id\":\"5e8970c3-26f7-4b8e-9c55-6c1c7f0b6d2e\",\"expires_at\":1791480600,\"configuration\":{\"voice_id\":\"aai-kyla\",\"language\":\"english\",\"sample_rate\":24000,\"encoding\":\"pcm_s16le\",\"word_boundaries\":true,\"model\":\"assemblyai-tts-1\",\"container\":\"raw\",\"bit_rate\":null,\"transport\":\"http\"}}`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`event: Audio`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`data: {\"type\":\"Audio\",\"audio\":\"AAAAAP7/AQD+/wIA\",\"flush_id\":0}`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`event: FlushDone`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`data: {\"type\":\"FlushDone\",\"flush_id\":0,\"audio_duration_ms\":1440}`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`event: WordBoundaries`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`data: {\"type\":\"WordBoundaries\",\"flush_id\":0,\"audio_start_ms\":0,\"words\":[{\"word\":\"Hello\",\"start\":265,\"end\":530,\"confidence\":0.48},{\"word\":\"there,\",\"start\":546,\"end\":712,\"confidence\":0.93},{\"word\":\"friend.\",\"start\":845,\"end\":1126,\"confidence\":0.85}]}`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`event: Termination`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`data: {\"type\":\"Termination\",\"session_duration_seconds\":1,\"total_input_char_length\":20,\"audio_duration_seconds\":1.44,\"billed_characters\":20,\"word_boundaries_status\":\"complete\"}`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})})]})})}),`\n`,_jsxs(_components.ul,{children:[`\n`,_jsxs(_components.li,{children:[`A successful stream is `,_jsx(_components.code,{children:`Begin`}),`, one or more `,_jsx(_components.code,{children:`Audio`}),`, `,_jsx(_components.code,{children:`FlushDone`}),`, usually one `,_jsx(_components.code,{children:`WordBoundaries`}),`, then `,_jsx(_components.code,{children:`Termination`}),`. As on `,_jsx(_components.code,{children:`/v1/speech`}),`, the status waits for the first audio, and `,_jsx(_components.code,{children:`Begin`}),` arrives with it.`]}),`\n`,_jsxs(_components.li,{children:[_jsx(_components.code,{children:`Audio`}),` carries base64 of raw `,_jsx(_components.code,{children:`pcm_s16le`}),`, `,_jsx(_components.code,{children:`pcm_mulaw`}),` or `,_jsx(_components.code,{children:`pcm_alaw`}),` audio, about 80 ms per event. MP3, WAV, `,_jsx(_components.code,{children:`bit_rate`}),` and `,_jsx(_components.code,{children:`\"stream\": false`}),` are refused on this route, because the word times are positions in that raw audio.`]}),`\n`,_jsxs(_components.li,{children:[_jsx(_components.code,{children:`WordBoundaries`}),` has one entry per whitespace-delimited token of your text, in order, with its punctuation, and `,_jsx(_components.code,{children:`start`}),` and `,_jsx(_components.code,{children:`end`}),` in milliseconds from the first sample. It follows the rules above: best effort, with zero-length entries for tokens that make no sound. `,_jsx(_components.code,{children:`Termination.word_boundaries_status`}),` says whether it was sent: `,_jsx(_components.code,{children:`complete`}),`, `,_jsx(_components.code,{children:`none`}),` when the timings could not be computed, or `,_jsx(_components.code,{children:`disabled`}),` when you sent `,_jsx(_components.code,{children:`\"word_boundaries\": false`}),`. `,_jsx(_components.code,{children:`Termination.billed_characters`}),` is the request’s final billed count.`]}),`\n`,_jsxs(_components.li,{children:[`A failure after the first audio sends one `,_jsx(_components.code,{children:`Error`}),` event, with the fields of the WebSocket `,_jsx(_components.code,{children:`Error`}),` frame plus `,_jsx(_components.code,{children:`code`}),`, `,_jsx(_components.code,{children:`message`}),`, `,_jsx(_components.code,{children:`request_id`}),` and `,_jsx(_components.code,{children:`status`}),`, and the stream ends. A stream that ends with neither `,_jsx(_components.code,{children:`Termination`}),` nor `,_jsx(_components.code,{children:`Error`}),` was cut, by the network or by a server failure: send the request again.`]}),`\n`,_jsxs(_components.li,{children:[`While nothing is sent for 15 seconds, a `,_jsx(_components.code,{children:`: keepalive`}),` comment arrives. Skip lines that start with `,_jsx(_components.code,{children:`:`}),`, and ignore event types and fields you do not know: `,_jsx(_components.code,{children:`Begin.configuration`}),` and `,_jsx(_components.code,{children:`Termination`}),` carry more fields than this section describes.`]}),`\n`]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`42`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`42`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` base64`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` json`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` os`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` httpx`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#D4D4D4`},children:`URL`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` =`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"https://tts.us.assemblyai.com/v1/speech/events\"`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#D4D4D4`},children:`HEADERS`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` =`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` {`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Authorization\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: os.environ[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"ASSEMBLYAI_API_KEY\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]}`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#D4D4D4`},children:`TEXT`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` =`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"Hello there, friend. The weather is lovely today.\"`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`audio `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#4EC9B0`},children:` bytearray`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`()`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`words `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` []`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`ended `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:` False`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`with`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` httpx.stream(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"POST\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#D4D4D4`},children:`URL`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#9CDCFE`},children:`headers`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#D4D4D4`},children:`HEADERS`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#9CDCFE`},children:`json`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"text\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#D4D4D4`},children:`TEXT`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}, `}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#9CDCFE`},children:`timeout`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`httpx.Timeout(`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`5.0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#9CDCFE`},children:`read`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`30.0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`)`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`) `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`as`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` response:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` if`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` response.status_code `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`!=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:` 200`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` response.read()`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` raise`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#4EC9B0`},children:` SystemExit`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`response.status_code`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`: `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`response.text[:`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`300`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`)`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` event `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:` None`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` for`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` line `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`in`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` response.iter_lines():`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` if`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` line.startswith(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\":\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`):`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` continue`}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:` # keepalive`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` if`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` line.startswith(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"event: \"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`):`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` event `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` line[`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#DCDCAA`},children:`len`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"event: \"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`):]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` elif`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` line.startswith(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"data: \"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`):`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` data `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` json.loads(line[`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#DCDCAA`},children:`len`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"data: \"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`):])`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` if`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` event `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`==`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"Audio\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` audio `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`+=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` base64.b64decode(data[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"audio\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`])`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` elif`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` event `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`==`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"WordBoundaries\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` words `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` data[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"words\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` elif`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` event `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`==`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"Termination\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` ended `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:` True`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` elif`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` event `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`==`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"Error\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` raise`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#4EC9B0`},children:` SystemExit`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`data[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`'code'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`: `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`data[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`'message'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` (`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`data[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`'request_id'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`)\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`)`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`if`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` not`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` ended:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` raise`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#4EC9B0`},children:` SystemExit`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"the stream was cut before its last event\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`)`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`with`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#DCDCAA`},children:` open`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"out.pcm\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"wb\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`) `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`as`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` f:`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` f.write(audio)`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`for`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` word `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`in`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` words:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#DCDCAA`},children:` print`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`word[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`'start'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`:>6}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` ms `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`word[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`'word'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`)`})]}),`\n`]})})}),`\n`,_jsx(Heading,{level:`2`,id:`related`,children:`Related`}),`\n`,_jsxs(_components.ul,{children:[`\n`,_jsxs(_components.li,{children:[_jsx(_components.a,{href:`/tts/connect-parameters`,children:`WebSocket API basics`}),`: reading `,_jsx(_components.code,{children:`word_boundaries`}),` back from the `,_jsx(_components.code,{children:`Begin`}),` echo.`]}),`\n`,_jsxs(_components.li,{children:[_jsx(_components.a,{href:`/tts/message-frames`,children:`Message frames`}),`: every frame, and where `,_jsx(_components.code,{children:`WordBoundaries`}),` sits in the order.`]}),`\n`]})]})}function MDXContent(props={}){const{wrapper:MDXLayout}={..._provideComponents(),...props.components};return MDXLayout?_jsx(MDXLayout,{...props,children:_jsx(_createMdxContent,{...props})}):_createMdxContent(props)}return{default:MDXContent};function _missingMdxReference(id,component){throw new Error(`Expected `+(component?`component`:`object`)+\" `\"+id+\"` to be defined: you likely forgot to import, pass, or provide it.\")}","frontmatter":{},"scope":{"config":{},"pageMetadata":{"title":"Word boundaries","description":"Per-word timings for synthesized speech, for highlighting text in sync with playback.","href":"/tts/word-boundaries"}}},"mdxSourceWithNoJs":null},"mdxExtracts":{"tableOfContents":[{"title":"The frame","slug":"the-frame","depth":2,"children":[{"title":"What word contains","slug":"what-word-contains","depth":3,"children":[]}]},{"title":"Times are on the session timeline","slug":"times-are-on-the-session-timeline","depth":2,"children":[]},{"title":"When it arrives","slug":"when-it-arrives","depth":2,"children":[]},{"title":"Mapping timings to playback","slug":"mapping-timings-to-playback","depth":2,"children":[]},{"title":"Treat them as best-effort","slug":"treat-them-as-best-effort","depth":2,"children":[]},{"title":"In the SDKs","slug":"in-the-sdks","depth":2,"children":[]},{"title":"Over the HTTP API","slug":"over-the-http-api","depth":2,"children":[]},{"title":"Related","slug":"related","depth":2,"children":[]}],"codeExamples":{}},"jsLinks":["/docs/_mintlify/custom/assemblyai-fff14a6f/83628fb7cd91f870/rudderstack.js","/docs/_mintlify/custom/assemblyai-fff14a6f/700d2574de973df9/chat-widget.js"],"subdomain":"assemblyai-fff14a6f","siteName":"AssemblyAI","title":"Word boundaries - AssemblyAI","navMaps":{"firstHrefInVersion":{},"firstHrefInLanguage":{}},"lastModified":"2026-11-06T20:50:32.264Z","originalFileLocation":"tts/word-boundaries.mdx"},"headHtml":"\u003cscript data-mint-head>(function(){var d=document.documentElement,p=null;if(!p){try{p=localStorage.getItem(\"isDarkMode\")}catch(e){}if(p==='true')p='dark';else if(p==='false')p='light';if(p!=='light'\u0026\u0026p!=='dark'\u0026\u0026p!=='system')p=\"system\"}var r=p==='system'?(matchMedia(\"(prefers-color-scheme: dark)\").matches?'dark':'light'):p;d.classList.remove(r==='dark'?'light':'dark');d.classList.add(r);d.setAttribute(\"data-theme-preference\",p)})();\u003c/script>\u003cmeta data-mint-head name=\"description\" content=\"Per-word timings for synthesized speech, for highlighting text in sync with playback.\">\u003cmeta data-mint-head name=\"application-name\" content=\"AssemblyAI\">\u003cmeta data-mint-head name=\"generator\" content=\"Mintlify\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/docs/mintlify-assets/_mintlify/favicons/assemblyai-fff14a6f/IP3FX8MjuVkHah_W/_generated/favicon/android-chrome-192x192.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"apple-touch-icon\" type=\"image/png\" sizes=\"180x180\" href=\"/docs/mintlify-assets/_mintlify/favicons/assemblyai-fff14a6f/IP3FX8MjuVkHah_W/_generated/favicon/apple-touch-icon.png\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/docs/mintlify-assets/_mintlify/favicons/assemblyai-fff14a6f/IP3FX8MjuVkHah_W/_generated/favicon/favicon-16x16.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/docs/mintlify-assets/_mintlify/favicons/assemblyai-fff14a6f/IP3FX8MjuVkHah_W/_generated/favicon/favicon-32x32.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/docs/mintlify-assets/_mintlify/favicons/assemblyai-fff14a6f/IP3FX8MjuVkHah_W/_generated/favicon/favicon.ico\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/docs/mintlify-assets/_mintlify/favicons/assemblyai-fff14a6f/IP3FX8MjuVkHah_W/_generated/favicon-dark/android-chrome-192x192.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/docs/mintlify-assets/_mintlify/favicons/assemblyai-fff14a6f/IP3FX8MjuVkHah_W/_generated/favicon-dark/favicon-16x16.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/docs/mintlify-assets/_mintlify/favicons/assemblyai-fff14a6f/IP3FX8MjuVkHah_W/_generated/favicon-dark/favicon-32x32.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/docs/mintlify-assets/_mintlify/favicons/assemblyai-fff14a6f/IP3FX8MjuVkHah_W/_generated/favicon-dark/favicon.ico\" media=\"(prefers-color-scheme: dark)\">\u003cmeta data-mint-head property=\"og:title\" content=\"Word boundaries - AssemblyAI\">\u003cmeta data-mint-head property=\"og:description\" content=\"Per-word timings for synthesized speech, for highlighting text in sync with playback.\">\u003cmeta data-mint-head property=\"og:site_name\" content=\"AssemblyAI\">\u003cmeta data-mint-head property=\"og:url\" content=\"https://www.assemblyai.com/docs/tts/word-boundaries\">\u003cmeta data-mint-head property=\"og:type\" content=\"website\">\u003cmeta data-mint-head property=\"og:image\" content=\"https://assemblyai-fff14a6f.mintlify.site/docs/_mintlify/api/og/v2?division=Features\u0026amp;title=Word+boundaries\u0026amp;description=Per-word+timings+for+synthesized+speech%2C+for+highlighting+text+in+sync+with+playback.\u0026amp;theme=7f291be5810d1f7c5136ed41\u0026amp;signature=7c295930a9945fabb5fed2e5d458771dc7758702949616e5f1f85efc8cd9687f\">\u003cmeta data-mint-head property=\"og:image:width\" content=\"1200\">\u003cmeta data-mint-head property=\"og:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"twitter:card\" content=\"summary_large_image\">\u003cmeta data-mint-head name=\"twitter:title\" content=\"Word boundaries - AssemblyAI\">\u003cmeta data-mint-head name=\"twitter:description\" content=\"Per-word timings for synthesized speech, for highlighting text in sync with playback.\">\u003cmeta data-mint-head name=\"twitter:image\" content=\"https://assemblyai-fff14a6f.mintlify.site/docs/_mintlify/api/og/v2?division=Features\u0026amp;title=Word+boundaries\u0026amp;description=Per-word+timings+for+synthesized+speech%2C+for+highlighting+text+in+sync+with+playback.\u0026amp;theme=7f291be5810d1f7c5136ed41\u0026amp;signature=7c295930a9945fabb5fed2e5d458771dc7758702949616e5f1f85efc8cd9687f\">\u003cmeta data-mint-head name=\"twitter:image:width\" content=\"1200\">\u003cmeta data-mint-head name=\"twitter:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"msapplication-config\" content=\"/docs/mintlify-assets/_mintlify/favicons/assemblyai-fff14a6f/IP3FX8MjuVkHah_W/_generated/favicon/browserconfig.xml\">\u003cmeta data-mint-head name=\"apple-mobile-web-app-title\" content=\"AssemblyAI\">\u003cmeta data-mint-head name=\"msapplication-TileColor\" content=\"#614fd2\">\u003clink data-mint-head rel=\"canonical\" href=\"https://www.assemblyai.com/docs/tts/word-boundaries\">\u003clink data-mint-head rel=\"alternate\" type=\"application/xml\" href=\"/docs/sitemap.xml\">\u003clink data-mint-head rel=\"alternate\" type=\"text/markdown\" href=\"/docs/tts/word-boundaries.md\">\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https://www.assemblyai.com/#organization\",\"name\":\"AssemblyAI\",\"url\":\"https://www.assemblyai.com\",\"logo\":{\"@type\":\"ImageObject\",\"url\":\"https://mintcdn.com/assemblyai-fff14a6f/vMuNaI_PbWUbVeV0/assets/assemblyai-lockup-primary.svg?fit=max\u0026auto=format\u0026n=vMuNaI_PbWUbVeV0\u0026q=85\u0026s=808d2a677e2536423045d44b3f330ac2\"}},{\"@type\":\"WebSite\",\"@id\":\"https://www.assemblyai.com/docs#website\",\"name\":\"AssemblyAI\",\"url\":\"https://www.assemblyai.com/docs\",\"publisher\":{\"@id\":\"https://www.assemblyai.com/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https://www.assemblyai.com/docs/tts/word-boundaries#webpage\",\"url\":\"https://www.assemblyai.com/docs/tts/word-boundaries\",\"name\":\"Word boundaries\",\"description\":\"Per-word timings for synthesized speech, for highlighting text in sync with playback.\",\"dateModified\":\"2026-11-06T20:50:32.264Z\",\"isPartOf\":{\"@id\":\"https://www.assemblyai.com/docs#website\"},\"breadcrumb\":{\"@id\":\"https://www.assemblyai.com/docs/tts/word-boundaries#breadcrumb\"}},{\"@type\":\"BreadcrumbList\",\"@id\":\"https://www.assemblyai.com/docs/tts/word-boundaries#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Features\",\"item\":\"https://www.assemblyai.com/docs/tts/voices\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Word boundaries\",\"item\":\"https://www.assemblyai.com/docs/tts/word-boundaries\"}]},{\"@type\":[\"Article\",\"TechArticle\"],\"@id\":\"https://www.assemblyai.com/docs/tts/word-boundaries#article\",\"headline\":\"Word boundaries\",\"name\":\"Word boundaries\",\"description\":\"Per-word timings for synthesized speech, for highlighting text in sync with playback.\",\"url\":\"https://www.assemblyai.com/docs/tts/word-boundaries\",\"mainEntityOfPage\":{\"@id\":\"https://www.assemblyai.com/docs/tts/word-boundaries#webpage\"},\"dateModified\":\"2026-11-06T20:50:32.264Z\",\"publisher\":{\"@id\":\"https://www.assemblyai.com/#organization\"},\"isPartOf\":{\"@id\":\"https://www.assemblyai.com/docs#website\"}}]}\u003c/script>","agentPointerHref":"/docs/tts/word-boundaries.md","siteName":"AssemblyAI","nav":{"global":null,"tabs":[{"tab":"Overview","pages":[{"title":"Overview","href":""}]},{"tab":"Pre-recorded STT","pages":[{"title":"Pre-recorded STT","href":"/pre-recorded-audio/getting-started/transcribe-an-audio-file"}]},{"tab":"Real-time STT","pages":[{"title":"Real-time STT","href":"/streaming/getting-started/transcribe-streaming-audio"}]},{"tab":"Sync STT","pages":[{"title":"Sync STT","href":"/sync-stt/getting-started/quickstart"}]},{"tab":"Dictation API","pages":[{"title":"Dictation API","href":"/dictation"}]},{"tab":"Text to Speech","groups":[{"group":"Getting started","pages":[{"title":"Quickstart","href":"/tts/getting-started/quickstart"},{"title":"Optimizing Accuracy and Latency","sidebarTitle":"Optimizing accuracy and latency","href":"/tts/getting-started/optimizing-accuracy-and-latency"},{"title":"Choose an interface","href":"/tts/getting-started/choose-an-interface"}]},{"group":"Features","pages":[{"title":"Voices","href":"/tts/voices"},{"title":"Voice cloning","href":"/tts/cloned-voices"},{"title":"Prompting guide","description":"How the text you send shapes the speech, from pacing and tone to how numbers, dates and other entities are read, and how to change it.","sidebarTitle":"Prompting","href":"/tts/prompting-guide"},{"title":"Word boundaries","href":"/tts/word-boundaries"}]},{"group":"Integrations","pages":[{"title":"Streaming TTS on LiveKit","description":"Give a LiveKit voice agent an AssemblyAI voice with the assemblyai.TTS plugin, with word-aligned transcripts and barge-in on one pooled session.","sidebarTitle":"LiveKit","href":"/tts/livekit"},{"title":"Streaming TTS on Pipecat","sidebarTitle":"Pipecat","href":"/tts/pipecat"},{"title":"Streaming TTS on OpenRouter","sidebarTitle":"OpenRouter","href":"/tts/openrouter"},{"title":"OpenAI compatibility","href":"/tts/openai-compatibility"}]},{"group":"API reference","pages":[{"group":"WebSocket API","pages":[{"title":"WebSocket API basics","href":"/tts/connect-parameters"},{"title":"Message frames","href":"/tts/message-frames"},{"title":"Streaming TTS WebSocket API","asyncapi":"api-reference/specs/streaming-tts.yaml tts","href":"/tts/api-reference/websocket-api"}]},{"group":"HTTP API","pages":[{"title":"HTTP API basics","href":"/tts/api-reference/http-api/overview"},{"title":"Create speech","openapi":"api-reference/specs/streaming-tts-speech.yaml POST /v1/speech","href":"/tts/api-reference/http-api/create-speech"},{"title":"Stream speech with word timings","openapi":"api-reference/specs/streaming-tts-speech.yaml POST /v1/speech/events","href":"/tts/api-reference/http-api/stream-speech-events"},{"title":"Create speech (OpenAI compatible)","openapi":"api-reference/specs/streaming-tts-speech.yaml POST /v1/openai/audio/speech","href":"/tts/api-reference/http-api/create-speech-openai"},{"title":"List models (OpenAI compatible)","openapi":"api-reference/specs/streaming-tts-speech.yaml GET /v1/openai/models","href":"/tts/api-reference/http-api/list-openai-models"},{"title":"List models (OpenRouter)","openapi":"api-reference/specs/streaming-tts-speech.yaml GET /v1/openrouter/models","href":"/tts/api-reference/http-api/list-openrouter-models"}]}]},{"group":"Advanced","pages":[{"title":"Authenticate with a temporary token","href":"/tts/authenticate-with-a-temporary-token"},{"title":"Error handling","href":"/tts/error-handling"},{"title":"Limits","href":"/tts/limits"},{"title":"Cloud endpoints \u0026 data residency","href":"/tts/endpoints-and-data-zones"}]}]},{"tab":"Voice Agent API","hidden":true,"pages":[{"title":"Voice Agent API","href":"/voice-agents/voice-agent-api"}]},{"tab":"Speech Understanding","pages":[{"title":"Speech Understanding","href":"/speech-understanding/getting-started"}]},{"tab":"Guardrails","pages":[{"title":"Guardrails","href":"/guardrails/getting-started"}]},{"tab":"LLM Gateway","pages":[{"title":"LLM Gateway","href":"/llm-gateway/quickstart"}]},{"tab":"FAQ","pages":[{"title":"FAQ","href":"/faq"}]}]},"basePath":"","title":"Word boundaries - AssemblyAI","buildId":"dpl_H5KQTnCewfWWvJxA7c3bCm7MFbb6","deploymentId":"dpl_H5KQTnCewfWWvJxA7c3bCm7MFbb6","dynamics":["10e4515597"],"chunks":["BannerMarkdown-BT--7wWI.js","dist-f_--J3AN.js","lib-DuNMoZ05.js","lib-B8HbPgcC.js","katex-YMM-FMeD.js","html-void-elements-mCSPHezU.js","marked.esm-CZWythHk.js"]}