Now live New models for transcription and text-to-speech. AssemblyAI-STT-3.7 AssemblyAI-TTS-1
Insights & Use Cases

Best voice agent API in 2026: 6 options compared for developers

Six voice agent APIs and platforms compared for developers: speech accuracy, turn-taking, public pricing and concurrency, with AssemblyAI-STT-3.7-Realtime-Agent.

Abstract green diamond illustration

Written by

Kelsey Foster

Published on

18 November 2026

The best voice agent API depends on how much of the stack you want to own. For developers building production agents, AssemblyAI supplies the full voice stack: AssemblyAI-STT-3.7-Realtime-Agent for streaming speech-to-text with turn detection built into the model, and the Text-to-Speech API for the voice, orchestrated with LiveKit, Pipecat, Vapi or your own loop, with the LLM of your choice. OpenAI Realtime, Deepgram, ElevenLabs, Vapi and Retell each fit different needs, compared below.

Most "best voice agent" lists rank options by how fast you get to a demo. That's useful, but it skips the question that decides production quality: does the agent hear your user correctly? A voice agent is a pipeline. Speech-to-text feeds the LLM, the LLM decides what to say, and the voice says it. If the first step gets a name or an account number wrong, everything downstream confidently answers a question the caller never asked.

So this guide compares six options on what holds up in production: accuracy on real agent audio, turn-taking, pricing model, concurrency and how much control you keep. Every competitor price below comes from the vendor's public pricing page, checked November 16, 2026.

Build it

Pick the speech model for your agent, set speech_model to assemblyai-stt-3-7-realtime-agent, and run it in LiveKit, Pipecat or your own loop.

Read the guide

How the six options compare

Option Type Pricing model (public, Nov 16, 2026) Concurrency Best for
AssemblyAI (STT-3.7-Realtime-Agent + Text-to-Speech API, with your orchestrator) Full voice stack: streaming speech-to-text, turn detection and text-to-speech; you choose the LLM and orchestrate with LiveKit, Pipecat, Vapi or your own loop $0.45/hr for STT-3.7-Realtime-Agent; Text-to-Speech API $15 per 1M characters; LLM billed on top Limits new sessions per minute, with no cap on open sessions (5/min free; 200/min to start on paid plans, growing 10% when you open 80% or more) Developers who want speech in, turn-taking and speech out from one vendor, with full control of the LLM and orchestration
OpenAI Realtime Managed API: speech-native multimodal models gpt-realtime-2.1: $32 / $64 per 1M audio input / output tokens; GPT-Live-1: $0.05/min plus backend model and tool usage Per-account limits Multimodal prototyping
Deepgram Voice Agent API Managed API on Deepgram's own STT Per minute of connection time: $0.075/min Standard, $0.163/min Advanced, from $0.050/min with your own LLM and TTS Capped per project; up to 60 (North America) or 45 (other regions) on Growth Teams already on Deepgram STT
ElevenLabs ElevenAgents Hosted agents platform Plans with included minutes; extra minutes $0.08/min 4–40 by plan; burst above the limit at $0.160/min A hosted agents platform with bundled minutes
Vapi Orchestration platform $0.05/min hosting, plus STT, LLM and voice at provider cost 4 on the usage plan; 10 on the $29/month Core package; more on higher packages Phone agents assembled from pass-through providers
Retell Phone-first agent platform $0.07–$0.31/min pay as you go, by component 20 included; $8 per extra concurrent call per month No-code phone agents for standard flows

Competitor pricing from public pricing pages as of November 16, 2026: OpenAI, Deepgram, ElevenLabs, Vapi, Retell. Carrier minutes are extra on every option. AssemblyAI's limits are from its streaming rate-limits docs.

One thing worth naming up front: LiveKit and Pipecat are open-source pipeline frameworks, not competitors on this list, and Vapi is both an option here and a place AssemblyAI runs. You can slot AssemblyAI in at both ends of an agent in those frameworks without rearchitecting anything.

What "best voice agent API" means in practice

The LLM gets the attention, but it never hears your customer. It reads a transcript a speech model produced a few hundred milliseconds earlier and treats it as ground truth. If the speech model hears "Nguyen" as "when," the agent doesn't know it misheard. It runs with it, and the caller has to repeat themselves.

A demo that works once in a quiet room tells you almost nothing. These are the criteria that separate an agent that ships from one that gets pulled after week one:

  • Speech accuracy on real audio. Names, account numbers, accents, cross-talk and background noise. Clean read-aloud clips don't count.
  • Turn detection. Knowing when the caller has finished versus pausing mid-thought. Get it wrong and the agent interrupts or sits in dead air.
  • Interruption handling. Stopping when the caller cuts in, but not on every "mhm."
  • Latency. Mouth to ear. Around a second keeps the conversation flowing.
  • Billing model. How many meters, and how each one moves with call length. This decides whether you can price a call before you scale.
  • Concurrency. Whether a Monday-morning spike hits a cap, a per-line fee or neither.
  • Developer experience. Plain protocols and readable docs, or an SDK and a conversation designer you have to bend your app around.

Weight these however your use case demands. But accuracy sits underneath most of them. Bad transcripts make turn detection harder, force retries that add latency, and create the "sorry, can you repeat that?" loops that wreck the experience.

Hear the speech layer before you build on it

Stream your own voice into AssemblyAI-STT-3.7-Realtime-Agent. Read it a hard account number, mumble a name, talk over background noise, and check what comes back. No setup required.

Try playground

Platform vs building blocks: which are you buying?

This is the fork in the road. Platforms like Vapi and Retell give you an opinionated way to design a conversation: hosted orchestration, a dashboard, and often telephony built in. You give up control of each layer, and your flow has to fit the platform's conversation model.

Building blocks go the other way. You pick the speech layer, the LLM and the orchestration, and you own the prompts, the tools and the integration. That's the path if you're building something differentiated or if accuracy is load-bearing. The two aren't exclusive: you can run AssemblyAI as the transcriber inside Vapi, or as the speech-to-text and the voice under a LiveKit or Pipecat agent you control end to end. We broke down the day-to-day side of this in which voice agent platform has the best developer experience.

The six options, one by one

Each one optimizes for something different.

AssemblyAI: the full voice stack

AssemblyAI covers every speech job in the agent. AssemblyAI-STT-3.7-Realtime-Agent hears the caller and decides when they've finished talking, the Text-to-Speech API speaks the reply, and LLM Gateway is there if you want the LLM on the same account. Your orchestrator (LiveKit, Pipecat, Vapi or your own loop) runs the conversation with whichever LLM you choose.

What's included

  • AssemblyAI-STT-3.7-Realtime-Agent at $0.45/hr, a speech-to-text model built specifically for voice agents, with keyterms prompting built in
  • Turn detection inside the speech model: at each pause it checks whether what's been said reads as a complete turn, and holds the turn open when it ends in an entity that may continue, like half a phone number. min_latency, balanced and max_accuracy presets, plus a manual mode (Agent-only) where your client ends each turn with ForceEndpoint
  • agent_context: pass the agent's last reply so the model expects the answer to that question
  • Mid-stream configuration updates for prompts, keyterms, turn timing and agent_context, without reconnecting
  • 32 languages with native code-switching
  • Voice Focus voice isolation as a $0.10/hr add-on
  • The Text-to-Speech API, built for conversational voice agents and trained on millions of hours of audio, including real conversational data: six languages, voice cloning, one WebSocket session per conversation that starts speaking from streamed LLM text, and cancel() for barge-in
  • LLM Gateway, one OpenAI-compatible API for GPT, Claude, Gemini and other models
  • Telephony audio at both ends: pcm_mulaw at 8 kHz into speech-to-text, and μ-law or A-law out of text-to-speech

What changed with STT 3.7. Against Universal-3.6 Pro Realtime, the previous model (still fully supported), the STT-3.7-Realtime-Agent research post reports 80% fewer number errors and 53% fewer entity misses on Voice Code Bench, 47% fewer words added while the user is silent, 38% fewer dropped one- and two-word replies, and 69% fewer words changing between partial and final transcripts. On a separate latency benchmark, the final transcript arrives 86 ms after the end of speech, against 172 ms.

OpenAI Realtime

OpenAI built multimodal models that also do voice. That's the framing to keep in mind. GPT-Live-1 adds another session shape alongside the Realtime API. The cost is a larger event surface to manage and billing that spreads across audio and text token meters, plus a per-minute session fee on GPT-Live-1 with backend model and tool usage charged separately. More in our roundup of OpenAI Realtime API alternatives.

Deepgram

Deepgram's Voice Agent API is a unified agent API built on its own speech models; Deepgram's docs list Deepgram as the only supported speech-to-text provider. The difference shows up in accuracy on real agent audio: on AssemblyAI's voice-agent benchmark, as of November 2026, entity error is 13.2% for AssemblyAI-STT-3.7-Realtime-Agent and 25.1% for Deepgram Nova-3. The full breakdown is in our AssemblyAI vs. Deepgram Voice Agent API comparison.

ElevenLabs

ElevenAgents is ElevenLabs’ hosted platform billed by call minutes on monthly plans, with LLM usage passed through separately. Concurrency is set by plan, from 4 to 40 concurrent calls, with burst capacity above that at double the per-minute rate. That's worth modeling if your traffic is spiky.

Vapi

Vapi is orchestration middleware. You configure assistants, bring the STT, LLM and voice providers you want, and pay a $0.05/min hosting fee plus each provider's cost. Your agent's accuracy, latency and cost are the sum of the providers you pick, plus the hosting fee. AssemblyAI is available as a Vapi transcriber. Vapi doesn't list STT-3.7-Realtime-Agent yet, so its AssemblyAI transcriber runs Universal-3.6 Pro Realtime, the previous model, which is still fully supported.

Retell

Retell is a phone-first platform with built-in telephony and pricing that adds up by component, from $0.07 to $0.31 per minute. You work inside its conversation model, so flows that need something it doesn't expose have nowhere to go.

The best voice agent API for each use case

Best for accuracy-critical production agents: AssemblyAI with your orchestrator

When names, order IDs and phone numbers decide whether the call succeeds, start with the speech model that has the lowest WER and entity error rate of the 26 streaming models we ran on AssemblyAI's voice-agent benchmark (runs August to November 2026), add the LLM you trust, and run it in LiveKit, Pipecat, Vapi or your own loop.

Best for multimodal prototyping: OpenAI Realtime

If your product mixes voice with images or text in one model session, OpenAI's realtime models keep them together, billed on token meters that move with how much the call says.

Best for a hosted agents platform with bundled minutes: ElevenLabs ElevenAgents

Plan-based minutes and a managed builder. Concurrency is capped by plan, and calls above the cap bill at double the per-minute rate, so model your peak before you pick a plan.

Best for no-code phone agents: Vapi or Retell

Both host the agent and the conversation design. Vapi passes provider costs through on top of its hosting fee (AssemblyAI is available as the transcriber); Retell bundles telephony and prices by component.

Best for teams assembling their own pipeline: LiveKit or Pipecat with AssemblyAI

If you need a specific LLM or a cloned voice, own every layer in a framework and plug in AssemblyAI for speech-to-text and the Text-to-Speech API for the voice. LiveKit's AssemblyAI plugin already accepts assemblyai-stt-3-7-realtime-agent; Pipecat doesn't list it yet and runs Universal-3.6 Pro Realtime in the meantime.

Best by vertical

Choosing only the speech-to-text layer? That's a narrower decision, covered in how to choose a speech-to-text API for voice agents. This guide is about the whole stack.

Why speech accuracy is the tiebreaker

When two options both promise low latency and a nice developer experience, accuracy on real audio is what's left to choose on.

AssemblyAI's English voice-agent benchmark runs 12,460 scripted caller scenarios, voiced by 522 real speakers and recorded as telephony audio with no, low or heavy background noise. Every provider is streamed through the same harness with no key terms or context, 26 streaming models in all, runs August to November 2026:

System (lower is better) Normalized WER Entity error rate
AssemblyAI-STT-3.7-Realtime-Agent 4.53% 13.2%
Soniox Realtime v6 6.02% 15.2%
ElevenLabs Scribe v3 Live 7.38% 16.9%
Deepgram Nova-3 8.64% 25.1%

AssemblyAI-STT-3.7-Realtime-Agent has the lowest WER and the lowest entity error rate of the 26 streaming models we ran, ahead of Soniox, ElevenLabs, Google, Speechmatics and Deepgram on both, and its lead over Soniox Realtime v6 holds in a paired 95% interval and at every noise level. It makes fewer errors than every other provider on each entity class the research post shows, phone numbers included (2.03%, against 2.20% for Soniox Realtime v6 and 4.35% for Deepgram Nova-3). It also has the fewest dropped short replies of the providers measured, and no other provider has both less background speech and fewer dropped short replies. Deepgram Flux EN posts a 13.50% WER and a 28.8% entity error rate on the same calls; the full tables are on the AssemblyAI benchmarks page.

Entity error rate matters even more for agents than WER. It measures how often the things that carry meaning come back wrong: names, account numbers, confirmation codes. When a caller reads back a card number, that's the metric deciding whether the agent charges the right account.

What you'll pay

With AssemblyAI as the speech layer, the speech-to-text line is $0.45/hr for AssemblyAI-STT-3.7-Realtime-Agent, pay as you go with no minimums. Voice Focus adds $0.10/hr if you turn it on. The Text-to-Speech API is $15 per 1M characters, billed on the input characters you send, and your LLM, any orchestration platform fee and your carrier minutes come on top.

That's more than one line item, but each one moves predictably. Speech-to-text scales with call duration, not with how chatty the context gets. Platforms and token-metered APIs bill on different meters, so the same call can cost more or less depending on how long and how talkative it runs. We priced the same call under each model in AI voice agent pricing, compared.

Before you commit: test it on your own calls

Run your 50 worst calls through every finalist: the accents, the noisy lines, the callers who spell their names. Measure entity error and P95 latency under realistic load, with many sessions at once. Then price cost per completed task and load test at your expected peak.

Getting started

The speech-to-text side is one WebSocket with standard JSON messages. Open a connection with your API key in the Authorization header and speech_model=assemblyai-stt-3-7-realtime-agent, stream audio in, and act on each Turn event where end_of_turn is true. The voice side is a second session on the same API key: the Python SDK (1.10.1 or later) opens it. The LiveKit integration handles both for you if you'd rather not touch the sockets.

Here's the shape of a minimal loop against the streaming WebSocket API. The LLM call and audio output are stubs for whatever you plug in, and the reply is spoken by the Text-to-Speech API:

import asyncio
import json
import os

import assemblyai as aai  # pip install -U "assemblyai>=1.10.1"
import websockets  # pip install websockets

API_KEY = os.environ["ASSEMBLYAI_API_KEY"]
URL = (
    "wss://streaming.assemblyai.com/v3/ws"
    "?speech_model=assemblyai-stt-3-7-realtime-agent"
    "&sample_rate=16000&encoding=pcm_s16le"
)
TTS_CONFIG = aai.TTSConfig(voice_id="aai-kyla", language="english")

async def llm(text):
    ...  # your LLM call (or LLM Gateway)

async def play(chunk):
    ...  # your audio output: raw 24 kHz 16-bit mono PCM

async def run(audio_chunks):
    headers = {"Authorization": API_KEY}
    async with websockets.connect(URL, additional_headers=headers) as ws:
        # One TTS session for the whole call, one request per agent turn
        async with aai.AsyncTTSSession(TTS_CONFIG, api_key=API_KEY) as tts:

            async def send_audio():
                async for chunk in audio_chunks:  # 50-1000 ms of 16 kHz PCM per message
                    await ws.send(chunk)
                await ws.send(json.dumps({"type": "Terminate"}))

            sender = asyncio.create_task(send_audio())
            async for message in ws:
                event = json.loads(message)
                if event["type"] == "Turn" and event["end_of_turn"]:
                    reply = await llm(event["transcript"])
                    # Give the speech model the agent's reply as context for the next turn
                    await ws.send(json.dumps({"type": "UpdateConfiguration", "agent_context": reply}))
                    async for chunk in tts.speak(reply):
                        await play(chunk)
                elif event["type"] == "Termination":
                    break
            await sender

The full loop, with a WAV file as the caller, LLM Gateway for the reply and one TTS session for the whole call, is in the companion repo.

Three details carry most of the quality. Send audio in 50 to 1,000 ms chunks at real-time pace. Update agent_context after every agent reply, because each update replaces the last one. And send Terminate when the call ends so the session closes cleanly. When the caller cuts in, call cancel() on the TTS session to stop the reply in flight. From there, move the loop into LiveKit when you need rooms, telephony or interruption handling around the TTS output.

The bottom line

Every option on this list will get you a demo. The differences show up on your worst calls, at your real volume, on an invoice someone has to forecast. Platforms hand you a hosted setup and take control of the layers in exchange. Frameworks leave the layers to you. Token-metered models bill on meters that move with how talkative the call gets.

But the agent can only act on what it heard. If accuracy is what your product lives or dies on, choose the speech layer first and build the rest on top of it.

Speech in and speech out from one vendor

AssemblyAI-STT-3.7-Realtime-Agent at $0.45/hr and the Text-to-Speech API on one API key. The free tier includes up to 333 hours of streaming speech-to-text, no credit card required.

Sign up free

Frequently Asked Questions (FAQs)

What is the best API for building voice agents?

For accuracy-critical production agents, the strongest setup is AssemblyAI's voice stack: AssemblyAI-STT-3.7-Realtime-Agent for streaming speech-to-text and turn detection and the Text-to-Speech API for voice output, orchestrated with LiveKit, Pipecat, Vapi or your own loop and paired with the LLM you choose. OpenAI Realtime suits multimodal prototyping, and Vapi and Retell suit no-code phone agents. The right pick depends on how much control you need and how much your calls depend on accuracy.

What is the best one-API solution for voice agents?

The closest thing to a one-API voice agent you still control is one vendor for both speech ends on one key: AssemblyAI-STT-3.7-Realtime-Agent for speech-to-text and turn detection, AssemblyAI Text-to-Speech for the reply, and LLM Gateway for the model in between. LiveKit, Pipecat or your own loop runs the conversation. You can swap the LLM without touching either speech layer, which all-in-one agent platforms usually don't allow.

Which voice agent API is best for multilingual customer support?

Pick a stack that follows a caller who switches languages mid-call instead of locking one language per session. AssemblyAI-STT-3.7-Realtime-Agent transcribes 32 languages with native code-switching, so a caller who moves between English and Spanish stays in one session. AssemblyAI Text-to-Speech speaks six languages (English, Spanish, French, German, Italian and Portuguese); set the language for each session to match the text your agent sends.

Which voice agent API has the best speech accuracy?

On AssemblyAI's English voice-agent benchmark (runs August to November 2026), AssemblyAI-STT-3.7-Realtime-Agent has a 4.53% word error rate and a 13.2% entity error rate, the lowest of the 26 streaming models we ran. Soniox Realtime v6 is at 6.02% and 15.2%, ElevenLabs Scribe v3 Live at 7.38% and 16.9%, and Deepgram Nova-3 at 8.64% and 25.1%. Entity error is the number to watch, because names and account numbers are what agents act on.

How much does a voice agent cost to run with AssemblyAI?

AssemblyAI-STT-3.7-Realtime-Agent is $0.45/hr, pay as you go with no minimums, and Voice Focus adds $0.10/hr if you use it. The Text-to-Speech API is $15 per 1M characters, billed on input characters, and your LLM, any orchestration platform fee and carrier minutes come on top. Compare cost per completed call on your own traffic, because call length, talk ratio and LLM choice move the total more than any single list rate.

Which voice agent API handles high call volume in production?

Choose an API whose limits grow with traffic instead of capping concurrent calls. AssemblyAI limits how many new streaming sessions you open per minute, with no cap on sessions already open: 200 to start on paid plans, rising 10% every minute you open 80% or more of the limit, with no concurrency fees or tier commitments. Load test at your expected peak before launch so you see P95 latency and error behavior under real traffic.