voice_id parameter on a WebSocket session and on an HTTP request, voice_id in the Python SDK’s TTSConfig, and voiceId on a JavaScript SDK session (voice_id in its HTTP requests, which take the API’s field names). It takes one of these:
- The library slug of an AssemblyAI voice, such as
aai-jane. - A voice’s ID, a UUID such as
49c443a4-8a1d-5f7e-b70b-98bdbd9823c3. - The slug of one of your account’s voices, such as a voice you cloned in the AssemblyAI dashboard.
voice_id out and you get aai-jane, the default voice.
- Python SDK
- JavaScript SDK
- Without an SDK
ASSEMBLYAI_API_KEY, or from aai.settings.api_key.Begin.configuration.voice_id on a session, which the SDKs expose on the Begin event, and the X-TTS-Voice-Id header on an HTTP response, which the SDKs expose on the result. It is the value you sent, with a UUID in lowercase, or aai-jane when you sent none. On a session, Begin.configuration.resolved_voice.voice_id gives the voice’s UUID, also when you named it by slug.
voice, which named a voice by a bare name such as voice=jane, is retired. A session or request that still sends it, with any value, is refused before any audio: 3006 on the WebSocket API and 422 on the HTTP API, with error_type invalid_parameter and param voice. When the value names one of the built-in voices, the message names the voice_id to send instead, such as voice_id=aai-jane.
AssemblyAI voices
The voice catalog in the AssemblyAI dashboard holds thousands of voices across the six languages the model speaks. Browse it there, listen to each voice, and copy itsvoice_id with Copy voice ID; the text-to-speech playground has the same action, which copies the voice_id its session sends. AssemblyAI’s voices carry slugs that begin with aai-. Each voice was recorded in one language, which the catalog shows, and speaks that language when you leave language out; see Voice and language.
These are good voices to start with. The examples on these pages use Audrey:
Built-in voices
Eleven AssemblyAI voices are built into the model:aai-jane, the default, aai-alba, aai-anna, aai-charles, aai-eve, aai-george, aai-jean, aai-mary, aai-michael, aai-paul and aai-vera. They speak English when you leave language out, and any of the six languages when you set it, though results are usually worse than in English, their own language; see Voice and language.
The first session with a voice
The built-in voices are always loaded. Any other voice, from the catalog or cloned, is loaded when a session with it opens, andBegin does not wait for it. The first session with a voice that has not been used recently therefore waits a little longer for its first audio when its text arrives right after Begin, and not at all when the voice has finished loading by the time the text arrives. Open the session when the call starts, before your first reply is ready, and keep it open for the conversation; see Optimizing accuracy and latency.
Voice and language
language is the language the session speaks. Set it to the language of the text you send. When you leave it out, a catalog voice or a cloned voice speaks its own language, the one it was recorded in, and a built-in voice, or a session that names no voice, speaks English. language also decides which model the session runs on: the default model of that language in the region, assemblyai-tts-1 for all six languages.
assemblyai-tts-1 speaks every voice in all six languages: set language and the voice speaks it, whatever language the voice was recorded in. To pick a voice recorded in the language of your text, such as Constanza for Mexican Spanish, look at the language the catalog shows for it.
- Python SDK
- JavaScript SDK
- Without an SDK
Begin.configuration.language names the language the session speaks, defaults included, and Begin.configuration.resolved_voice.language the language the voice was recorded in. On the HTTP API, the X-TTS-Language header names the language of the request.
language takes the full English name of the language, such as spanish, not a code such as es, in any letter case. This differs from the STT APIs, which take short language codes such as es or en_us. The OpenAI-compatible route and the LiveKit plugin accept an ISO 639-1 code such as es as well.
Languages
language takes one of these values:
A session speaks a language only where a model of the region speaks it. Both data zones serve all six languages on one model,
assemblyai-tts-1, the default model of each of them, and Begin.configuration.model names the model a session runs on. A language outside this table closes the session with 3006 before Begin, with error_type invalid_parameter and param language, and the text lists the supported values. A supported language that no model of the region speaks closes with 3006, error_type invalid_request, param language and the text No voice model serves language <language> in this region.
What a wrong voice looks like
The server checks the voice when the session opens. A voice it cannot use never getsBegin: an Error frame arrives in its place, and the socket closes with the frame’s error_code. The Python SDK raises it as TTSError from the with block, the JavaScript SDK rejects connect() with a TextToSpeechError; both carry the frame’s error_type as error_code (errorCode) and its param. On the HTTP API the same refusals are 422 answers with the same codes, which the SDKs raise from synthesize() and stream().
- No voice with this
voice_id. Avoice_idthat names no voice your account can use on this host, such as a cloned voice that was deleted, closes the session with3012,error_typevoice_not_foundandparamvoice_id. A bare name such asvoice_id=janeis not a library slug: it is looked up among the slugs of your own voices, and unless one of your voices has that slug, it is not found either, with a message that endsDid you mean voice_id=aai-jane?. Sendaai-jane. - A built-in voice the session’s model does not speak. A built-in voice that the session’s model does not speak in the session’s language closes the session with
3006,error_typeinvalid_requestandparamvoice_id, and the message lists the voices available on that connection by their slugs.
voice_id that names no voice (voice_id=aai-nonexistent):
3006 whose param is voice_id after Begin as well: in rare cases the refusal of a built-in voice the model does not speak arrives right after Begin, with the same text, before any audio. Read configuration.voice_id and configuration.language from Begin to confirm what the session resolved, defaults included. See Error handling.
Related
- Voice cloning: a voice you clone in the AssemblyAI dashboard, named by its
voice_id. - WebSocket API basics: the connect sequence and the rules for the rest of the query string.
- Error handling: what to do with
3006,3012and the other close codes. - Optimizing accuracy and latency: open the session before the first reply.
- Quickstart: a full session using Audrey.