{
"type": "Generate",
"text": "Hi, thanks for calling. "
}{
"type": "Flush"
}{
"type": "Cancel"
}{
"type": "KeepAlive"
}{
"type": "Terminate"
}{
"type": "Begin",
"id": "10885144-0d4b-4928-9eff-ef292a352937",
"expires_at": 1789556741,
"configuration": {
"language": "english",
"sample_rate": 24000,
"encoding": "pcm_s16le",
"inactivity_timeout": 120,
"word_boundaries": true,
"voice_id": "aai-jane",
"extend_biometric_consent": false,
"word_boundaries_requested": true,
"model": "assemblyai-tts-1",
"voice_expires_at": null,
"session_ttl_sec": 3600,
"limits": {
"max_generate_text_length": 2000,
"max_control_frames_per_sec": 20,
"max_egress_audio_buffer_ms": 300000,
"max_egress_audio_buffer_bytes": 16777216,
"text_throttle_enabled": true,
"text_throttle_max_chars_per_sec": 100,
"text_throttle_initial_grace_chars": 1500,
"text_max_buffer_chars": 8000,
"session_ttl_grace_sec": 60,
"max_message_bytes": 1048576,
"max_queued_messages": 10000,
"write_stall_timeout_sec": 30,
"inactivity_timeout_min_sec": 5,
"inactivity_timeout_max_sec": 120
},
"unknown_params": []
}
}{
"type": "Audio",
"audio": "IwB6AOMABQHHAIMAJADw/wwA",
"flush_id": 0
}{
"type": "FlushDone",
"flush_id": 0,
"audio_duration_ms": 2320
}{
"type": "WordBoundaries",
"flush_id": 0,
"audio_start_ms": 0,
"words": [
{
"word": "Hello",
"start": 265,
"end": 530,
"confidence": 0.4791773557662964
},
{
"word": "there,",
"start": 546,
"end": 712,
"confidence": 0.9253882169723511
}
]
}{
"type": "Cancelled",
"audio_duration_seconds": 0.4
}{
"type": "Termination",
"session_duration_seconds": 12,
"total_input_char_length": 96,
"audio_duration_seconds": 3.12,
"voice_expires_at": null
}{
"type": "Error",
"error_code": 3009,
"error": "Too many sessions opened for this account in the last minute (allowance 100 per minute); retry in 17 s.",
"error_type": "rate_limited",
"retryable": true,
"param": null,
"retry_after_seconds": 17
}Streaming TTS WebSocket API
Send text and receive synthesized speech. Authenticate with your API key or a temporary token in the Authorization header, or with the token query parameter where headers can’t be set.
{
"type": "Generate",
"text": "Hi, thanks for calling. "
}{
"type": "Flush"
}{
"type": "Cancel"
}{
"type": "KeepAlive"
}{
"type": "Terminate"
}{
"type": "Begin",
"id": "10885144-0d4b-4928-9eff-ef292a352937",
"expires_at": 1789556741,
"configuration": {
"language": "english",
"sample_rate": 24000,
"encoding": "pcm_s16le",
"inactivity_timeout": 120,
"word_boundaries": true,
"voice_id": "aai-jane",
"extend_biometric_consent": false,
"word_boundaries_requested": true,
"model": "assemblyai-tts-1",
"voice_expires_at": null,
"session_ttl_sec": 3600,
"limits": {
"max_generate_text_length": 2000,
"max_control_frames_per_sec": 20,
"max_egress_audio_buffer_ms": 300000,
"max_egress_audio_buffer_bytes": 16777216,
"text_throttle_enabled": true,
"text_throttle_max_chars_per_sec": 100,
"text_throttle_initial_grace_chars": 1500,
"text_max_buffer_chars": 8000,
"session_ttl_grace_sec": 60,
"max_message_bytes": 1048576,
"max_queued_messages": 10000,
"write_stall_timeout_sec": 30,
"inactivity_timeout_min_sec": 5,
"inactivity_timeout_max_sec": 120
},
"unknown_params": []
}
}{
"type": "Audio",
"audio": "IwB6AOMABQHHAIMAJADw/wwA",
"flush_id": 0
}{
"type": "FlushDone",
"flush_id": 0,
"audio_duration_ms": 2320
}{
"type": "WordBoundaries",
"flush_id": 0,
"audio_start_ms": 0,
"words": [
{
"word": "Hello",
"start": 265,
"end": 530,
"confidence": 0.4791773557662964
},
{
"word": "there,",
"start": 546,
"end": 712,
"confidence": 0.9253882169723511
}
]
}{
"type": "Cancelled",
"audio_duration_seconds": 0.4
}{
"type": "Termination",
"session_duration_seconds": 12,
"total_input_char_length": 96,
"audio_duration_seconds": 3.12,
"voice_expires_at": null
}{
"type": "Error",
"error_code": 3009,
"error": "Too many sessions opened for this account in the last minute (allowance 100 per minute); retry in 17 s.",
"error_type": "rate_limited",
"retryable": true,
"param": null,
"retry_after_seconds": 17
}Optional. Authenticate with a temporary token (minted with product=tts) or an API key, for clients that cannot set the Authorization header, such as browsers. Ignored when the Authorization header carries a credential; a header that is empty or only Bearer carries none. Mint one temporary token per connection, and never put your API key in a URL that runs on a user's device.
Optional. The voice: the library slug of an AssemblyAI voice such as aai-jane, a voice's ID (a UUID, in any letter case), or the slug of one of your account's voices, such as a cloned voice. Slugs are lowercase, at most 64 characters. When unset, the session speaks the default voice of its model in its language, aai-jane, an English voice; in another language, results are usually worse than in English, its own language (see Voice and language). An AssemblyAI voice the session's model does not speak in the session's language closes with 3006; a voice_id that names no voice your account can use on this host closes with 3012.
Optional. The model to run the session on, such as assemblyai-tts-1, in lowercase; latest names the default model of the session's language. When unset, the default model of the session's language in the region. A cloned voice in voice_id must be paired with that model (3014 otherwise). A model the region does not know, or one that does not speak the session's language, closes with 3006 before Begin. See Language and model.
Optional. The language of the text you send, as its full English name, not an ISO code. Case-insensitive, echoed lowercase. The API does not detect language. Defaults to the language of the voice in voice_id, and to english when you name no voice. A value outside this list closes the session with 3006 before Begin, and so does a language the voice cannot speak on the session's model (never on assemblyai-tts-1, which speaks every voice in all six languages, though results are usually worse than in the voice's own language; see Voice and language). See Languages.
english, spanish, german, italian, portuguese, frenchOptional. Output sample rate in Hz. 24000 is the model's native rate; other rates are resampled.
8000, 16000, 22050, 24000, 44100, 48000Optional. Encoding of the audio in Audio messages: raw bytes with no header. pcm_s16le is 16-bit signed little-endian PCM; pcm_mulaw and pcm_alaw are 8-bit G.711 for telephony. The aliases mulaw and alaw are accepted and echoed in canonical form. See Audio formats on WebSocket API basics.
pcm_s16le, pcm_mulaw, pcm_alawOptional. Whether to send a WordBoundaries message with per-word timings after each request's FlushDone. Takes true, false, 1 or 0, in any case. See Word boundaries.
true, falseOptional. Seconds with no frame in either direction before the session closes with 3006 (error_type inactivity_timeout). Shortens the idle timeout every session has. An integer from 5 to 120; Begin.configuration.limits reports the accepted range.
Optional. With voice_id, by the owner of a cloned voice only, true renews the voice's biometric consent and attests renewed consent on behalf of the account that owns it. Set on a voice your account does not own, an AssemblyAI voice included, the session closes with 3006 (error_type not_owner), and set without voice_id, with 3006 (error_type invalid_parameter). The account must have accepted the biometric data terms (3006 biometric_terms_not_accepted otherwise). See Renew consent.
true, falseClient message with text to speak.
Client message that concludes the current request.
Client message that stops the speech in flight, for barge-in.
Client message that restarts the idle timer.
Client message that ends the session cleanly.
Server message confirming the session opened, with its effective configuration.
Server message with a chunk of synthesized audio.
Server message confirming a request's audio has all been sent.
Server message with per-word timings for a request.
Server message answering a Cancel.
Server message with the session summary after a clean Terminate.
Server message sent before every failure close, with the reason.
Was this page helpful?