Updated November 2026. Ranks come from our public benchmarks page, including the categories where AssemblyAI trails. Streaming prices come from each vendor's own pricing page, dated.
If you're picking a real-time speech-to-text API in 2026, three questions decide it: how accurate is it on the words that matter (names, numbers, account IDs), what does it cost per streaming hour, and how does it behave once thousands of WebSocket sessions are open at the same time. This guide answers all three for the major cloud APIs and the main open-source options.
Our pick for production voice agents and contact centers is AssemblyAI's Universal-3.6 Pro Realtime. On the public benchmarks page, it has the lowest voice-agent entity error rate of 22 real-time systems, at $0.45/hr. It isn't the fastest model on every leaderboard, and it doesn't win every category. The tables below show where it trails, too.
Want a general framework for evaluating any speech-to-text API? Start with how to choose a speech-to-text API. Building a voice agent specifically? The voice-agent evaluation guide walks through the seven tests that matter for agents.
What is real-time speech recognition?
Real-time speech recognition, also called streaming speech-to-text, converts live audio into text while the person is still talking. Your application opens a persistent WebSocket connection, sends audio in small chunks, and receives transcripts back continuously.
That's different from batch (async) transcription, which needs the whole file before it starts:
- Batch: upload a recording, get the full transcript when it's done. Good for meetings, podcasts and call archives.
- Real-time: text arrives within a fraction of a second of the words being spoken. Required for voice agents, live captions and agent assist.
One note before the comparison: not every "real-time" need is a live session. If you're transcribing short, self-contained clips, like a voice command or an IVR prompt, the Sync API returns a finished transcript from one HTTP POST in about 134 ms at the median, with no WebSocket to manage.
How streaming speech-to-text works
Every streaming API follows the same four steps:
| Stage | What happens | Result |
|---|---|---|
| Connect | Your app opens a WebSocket to the transcription server | A persistent two-way connection |
| Stream | Audio is sent in small chunks as it's captured | A continuous flow of audio |
| Transcribe | The model processes each chunk as it arrives | Partial and final transcripts |
| Deliver | Results come back over the same connection | Text your app can display or act on |
Two things separate a good streaming model from a mediocre one.
Turn detection. The model has to decide when a speaker has finished. End a turn too early and your agent interrupts the caller halfway through a phone number. End it too late and the agent feels slow. Universal-3.6 Pro Realtime uses entity-aware endpointing, so it holds a turn open while someone reads out a number or code.
Context. Real calls are full of follow-ups that only make sense given what was said a moment ago. Universal-3.6 Pro Realtime carries the last five turns forward automatically, and you can pass the agent's last reply with agent_context.
Real-world applications
- AI voice agents. Agents need accurate text fast enough to keep the conversation natural, and they need the entities right. In AssemblyAI's voice agent report, 95% of respondents said they've been frustrated with a voice agent at some point, and the top frustration was having to repeat themselves.
- Contact centers and agent assist. Live transcripts drive real-time coaching, compliance prompts and knowledge lookups during the call. Accuracy in noisy, overlapping speech matters more than shaving milliseconds.
- Live captioning. Captions are judged on readability, so formatting and punctuation matter as much as speed.
- Voice commands. Short, discrete commands need quick responses. For one-shot commands without an open session, the Sync API is often the simpler fit.
Quick comparison: real-time speech-to-text APIs in 2026
Here's how the main options compare on streaming price, languages and fit. Prices are each vendor's published pay-as-you-go streaming rate, converted to per hour.
| Solution | Type | Streaming price | Languages (streaming) | Best for |
|---|---|---|---|---|
| AssemblyAI Universal-3.6 Pro Realtime | Cloud API | $0.45/hr base, keyterms included | 32, with automatic detection and mid-sentence code-switching | Voice agents, contact centers, noisy and multilingual audio |
| AssemblyAI Universal-Streaming (English and Multilingual) | Cloud API | $0.15/hr (keyterms +$0.04/hr) | English; Multilingual covers EN, ES, DE, FR, PT, IT | High-volume streaming where the flagship's accuracy isn't required |
| Deepgram Nova-3 (monolingual) | Cloud API | $0.0077/min ($0.462/hr) regular rate | See Deepgram docs | Teams already on Deepgram |
| Deepgram Flux (English) | Cloud API | $0.0077/min ($0.462/hr) regular rate | English only | English-only voice agents |
| ElevenLabs Scribe v2 Realtime | Cloud API | $0.39/hr; keyterm prompting +$0.08/hr | See ElevenLabs docs | Teams already using ElevenLabs |
| Gladia real-time | Cloud API | €0.68/hr | See Gladia docs | EUR-billed teams |
| Google Cloud Speech-to-Text V2 | Cloud API | $0.016/min ($0.96/hr), first 500,000 min/month | See Google docs | Apps already on Google Cloud |
| OpenAI gpt-live-transcribe | Cloud API | $0.017/min ($1.02/hr), listed as estimated cost | See OpenAI docs | Apps already on OpenAI |
| Speechmatics real-time | Cloud API | See Speechmatics pricing page | See Speechmatics docs | Teams needing on-prem options |
| WhisperX and Whisper Streaming | Open source | Your infrastructure cost | Whisper's language set | Self-hosted control and research |
Prices as of November 10, 2026, from each vendor's own pricing page: AssemblyAI, Deepgram, ElevenLabs, Gladia, Google Cloud, OpenAI and Speechmatics. Deepgram also lists lower "limited-time promotional" streaming rates; the table shows its regular rates. For the full rate cards, including pre-recorded and add-ons, see the speech-to-text API pricing guide.
Talk to Universal-3.6 Pro Realtime live or upload a call, and check how it handles names, numbers and account IDs before you shortlist anything.
Real-time speech recognition accuracy benchmarks (2026)
Accuracy is where streaming APIs differ most, and averages hide the difference. A model can post a decent word error rate and still miss the one account number that decides whether the call succeeds.
Where Universal-3.6 Pro Realtime ranks
The benchmarks page runs Universal-3.6 Pro Realtime against 26 competitor models through each vendor's public streaming API at default settings (realtime snapshot dated 2026-09-16). Lower is better on every metric.
| Metric | Universal-3.6 Pro Realtime | Rank | Nearest competitor |
|---|---|---|---|
| Voice-agent entity error | 14.41% | 1st of 22 | Soniox Realtime, 18.19% |
| Judge-scored entity error | 18.93% | 2nd of 22 | Google Gemini 3.5 Live, 19.68%* |
| Code-switching WER (5 pairs) | 7.20% | 1st of 23 | AWS Transcribe, 9.13% |
| English short-form WER | 3.44% | 1st of 22 | Reson8, 3.70% |
| Voice-agent WER, heavy noise | 6.13% | 1st of 22 | Soniox Realtime, 7.68% |
| Voice-agent WER, Indian-accented English | 6.64% | 1st of 22 | Soniox Realtime, 7.78% |
| Medical entity error | 4.24% | 2nd of 22 | Meta Muse Voice Transcribe leads at 4.08% |
| Diarization error rate | 16.68% | 3rd of 12 | Meta Muse Voice Transcribe leads at 14.51% |
| Multilingual WER (FLEURS) | 8.44% over 18 languages | 5th of 22 | Reson8, 5.77% over 7 languages |
| Long-form English WER | 8.82% | 6th of 22 | Meta Muse Voice Transcribe leads at 7.37% |
| Non-speech response rate | 0.27% | 9th of 22 | Mistral Voxtral Mini leads at 0.03% |
*The only system ahead of Universal-3.6 Pro Realtime on judge-scored entity error is the previous AssemblyAI generation, Universal-3.5 Pro Realtime, at 18.1%. Gemini 3.5 Live is the nearest competing vendor.
The pattern is clear. Universal-3.6 Pro Realtime leads on the things voice agents depend on: entities, noise, accents, code-switching and short turns. It trails on long-form English, diarization, multilingual FLEURS and non-speech handling. If your workload is long monologues or speaker-heavy meetings, weigh those rows more.
How the field compares on voice-agent entity error
Entity error is the share of names, places, addresses, phone numbers, emails, dates, numbers and codes that come out wrong. Here's the overall figure for each vendor's best streaming model on the same voice-agent suite, from the same /benchmarks run:
| Streaming model | Voice-agent entity error |
|---|---|
| AssemblyAI Universal-3.6 Pro Realtime | 14.4% |
| Soniox Realtime | 18.2% |
| ElevenLabs Scribe v2 | 18.5% |
| AWS Transcribe | 19.3% |
| Google Gemini 3.5 Live | 20.1% |
| Speechmatics Enhanced | 20.6% |
| Deepgram Nova-3 | 26.1% |
| Azure | 29.7% |
| Deepgram Flux EN | 30.1% |
| Gladia Solaria | 30.6% |
| OpenAI GPT-4o Transcribe | 38.1% |
Two things stand out. Azure and Deepgram Flux EN miss roughly twice as many entities as Universal-3.6 Pro Realtime on this suite. And Google's best streaming entry here is Gemini 3.5 Live, not the Cloud Speech-to-Text V2 API priced in the table above, so test the exact Google product you plan to deploy. On the launch benchmark of 12,460 voice-agent scenarios, Universal-3.6 Pro Realtime also posts a 5.19% WER. The per-entity-class breakdown for agents (phone numbers, emails, codes) is in the voice-agent evaluation guide linked above.
Independent benchmarks and latency
Outside leaderboards test streaming models on shared audio, and they don't always agree. The benchmarks page reproduces three of them:
- pipecat-ai/stt-benchmark (open source, read 2026-09-28): Universal-3.6 Pro Realtime posts 0.96% pooled semantic WER, 2nd of 23, with a 307 ms median time to final transcript. Several models return finals sooner on that table, such as NVIDIA Nemotron 3.0 ASR at 221 ms and Deepgram Nova-3 at 247 ms, at higher WER.
- Cekura Converse-STT (updated 2026-09-16): Universal-3.6 Pro Realtime ranks 1st of 19 on clean-clip WER at 1.77%. On the small real-call set, it ranks 14th of 19.
- Artificial Analysis: the page reproduces ranks for the previous generation, Universal-3.5 Pro Realtime, which places 3rd to 12th of 31 depending on the test set.
Be clear-eyed about latency: Universal-3.6 Pro Realtime is built to be the most accurate streaming model, not the fastest. On AssemblyAI's own voice-agent benchmark, its median endpoint latency is 537 ms. For how latency is measured, see the real-time STT latency benchmarks.
Universal-3.6 Pro Realtime: AssemblyAI's flagship real-time model
Universal-3.6 Pro Realtime (universal-3-6-pro) is AssemblyAI's default streaming model. Switching from Universal-3.5 Pro Realtime is a one-line change to speech_model.
What's in it:
- 32 languages with mid-sentence code-switching. Callers who drop a Spanish phrase into an English sentence get transcribed correctly in one stream.
- Entity-aware endpointing. Turn detection holds a turn open while a caller reads out a phone number or code, so the number comes back in one piece.
- Context carryover. The last five turns carry forward automatically, and
agent_contextlets you pass what the agent just said. - Voice Focus.
voice_focusisolates the primary speaker in near-field or far-field audio. - Streaming diarization with revisions. Set
speaker_labels: truefor inline speaker labels, revised mid-session as more context arrives. - Keyterms included. Boost up to 100 domain terms, like product names or medications, and update them mid-stream.
Teams that evaluated the previous generation against their live pipelines reported:
"We were searching for the best realtime ASR model for our voice agent pipeline in Fireflies. The new Universal 3.5 Pro speech model from Assembly is best so far in terms of accuracy, latency and language switching."
— Foysal Osmany, Software Engineer at Fireflies
Pricing: $0.45/hr base, pay-as-you-go with no minimum commitment. Keyterms are included. Optional add-ons: streaming diarization +$0.12/hr, prompting +$0.05/hr, Voice Focus +$0.10/hr, Medical Mode +$0.15/hr and PII Text Redaction +$0.12/hr.
Integration: connect to wss://streaming.assemblyai.com/v3/ws and set speech_model to universal-3-6-pro. It's also available through the LiveKit and Pipecat integrations. See the streaming speech-to-text product page for details.
Building a full voice agent? Pair Universal-3.6 Pro Realtime with LLM Gateway for reasoning and AssemblyAI-TTS-1 for the voice, on one API key. AssemblyAI-TTS-1 is trained on millions of hours of audio, including real conversational data, and costs $15 per 1M characters.
Legacy model IDs. If your code still pins an older Universal-3 Pro streaming model ID, switch to universal-3-6-pro. Universal-3.5 Pro Realtime (universal-3-5-pro) stays available if you need to pin the previous model.
How streaming APIs behave in production: billing, limits and reconnects
Rate cards compare the price of an hour. Production bills depend on how the connection behaves. These are the mechanics to check for any streaming API before you sign up, with AssemblyAI's documented behavior as the reference.
Billing: session time, not audio time
AssemblyAI streaming is billed on how long the WebSocket stays open, not on how much audio you send. A session that sits idle while your agent waits on an LLM still counts. Send a termination message when the call ends; sessions that are never closed auto-close after 3 hours and bill for the full duration. Ask every vendor the same question, because "per minute of audio" and "per minute of connection" can differ a lot for agents with long silences.
Session length and reconnects
A single session runs for up to 3 hours. When it hits the limit, the server closes it with code 3008 after the current turn ends. Context carryover is per session, so a reconnect starts fresh: re-send agent_context and your keyterms on the new connection. If you mint temporary tokens for browser clients, max_session_duration_seconds lets you set a shorter cap.
Sending audio faster than real time
You can stream pre-recorded files for testing, but throughput is throttled at about 1.25x real time. If more than five minutes of audio buffers ahead of processing, the session closes with error 3007. The same code fires for chunks shorter than 50 ms or longer than 1,000 ms, so keep PCM chunks around 50 ms.
Concurrency and rate limits
AssemblyAI has no concurrency fee and no spend commitment to unlock scale. The limit is on new sessions per minute: pay-as-you-go accounts start at 100, and the limit rises automatically as you use more of it. Exceed it and the server closes the new connection with code 1008 and the message "Too many concurrent sessions"; back off and retry in the next minute. Compare that with plan-capped concurrency on other vendors. Gladia's self-serve plan, for example, caps you at 30 concurrent real-time sessions.
Sign up free, swap the model into your pipeline, and compare entity accuracy on the audio your product actually handles.
Key criteria for selecting a real-time speech recognition API
The right API depends on your application's constraints. Here's what matters most for real-time work.
- Latency budget. Voice agents need speech recognition, the LLM and text-to-speech to fit inside a natural conversational pause. Decide your end-to-end budget first, then see how much of it speech recognition can take. A fast model that mishears the account number forces a repeat, which costs far more time than a few hundred milliseconds.
- Accuracy on the words that matter. Average word error rate hides what breaks production: names, numbers, emails and IDs. Look for entity error rates, and test on audio that looks like yours. For why headline numbers mislead, see why word error rate is broken.
- Language support. Large language counts on a marketing page don't mean every language is production-ready in streaming. Test your target languages, and check whether the model handles callers who switch languages mid-sentence.
- Production behavior. Session billing, reconnects and rate limits, covered above.
- Integration effort. Cloud APIs ship faster but add an external dependency. Open-source models give you control but demand real engineering for streaming, endpointing, scaling and security.
Other cloud APIs
Deepgram Nova-3 and Flux
Deepgram sells Nova-3 for general streaming and Flux for English voice agents. On the /benchmarks voice-agent suite, Nova-3 posts 26.1% entity error and Flux EN 30.1%, against 14.4% for Universal-3.6 Pro Realtime. Flux's published rate covers English only, and Deepgram's lower streaming rates are labeled limited-time promotions. Pay-as-you-go concurrency is capped by plan.
ElevenLabs Scribe v2 Realtime
ElevenLabs Scribe v2 Realtime is $0.39/hr on its API pricing page, and keyterm prompting is a paid add-on at $0.08/hr. It posts 18.5% voice-agent entity error on /benchmarks.
Gladia
Gladia's real-time API is priced at €0.68/hr, billed in euros. Its self-serve plan caps you at 30 concurrent real-time sessions, and requests above the limit get a 429 response. Gladia Solaria posts 30.6% voice-agent entity error on /benchmarks.
Google Cloud Speech-to-Text
Google's V2 standard recognition is $0.016/min ($0.96/hr) for the first 500,000 minutes a month. It sits inside a pipeline of other billed Google Cloud services, such as storage and functions. Google's best streaming entry on /benchmarks, Gemini 3.5 Live, posts 20.1% voice-agent entity error.
OpenAI
OpenAI lists two live transcription models, gpt-live-transcribe and gpt-realtime-whisper, at an estimated $0.017/min ($1.02/hr). That's the highest streaming rate in this comparison. GPT-4o Transcribe posts 38.1% voice-agent entity error on /benchmarks.
AWS Transcribe and Microsoft Azure
Both offer streaming transcription inside their clouds. On /benchmarks, AWS Transcribe posts 19.3% voice-agent entity error and Azure 29.7%. Azure's realtime diarization error rate is 20.43%, 5th of 12, against 16.68% for Universal-3.6 Pro Realtime. Check each provider's pricing page for current streaming rates.
Open-source models
WhisperX
WhisperX extends OpenAI's Whisper with batched inference, word-level timestamps and speaker diarization. It wasn't built for live streaming, so real-time use needs extra engineering for chunking, endpointing and scaling. For the cost side of self-hosting, see the total cost of ownership breakdown.
Whisper Streaming
Whisper Streaming builds a real-time wrapper around Whisper. It's useful for research and prototypes. Production use means GPU capacity planning, latency tuning and your own turn detection.
Which real-time API should you choose?
- Production voice agents and contact centers: Universal-3.6 Pro Realtime. It has the lowest voice-agent entity error rate on the benchmarks page, 32 languages with code-switching, Voice Focus for noisy audio, and keyterms included at $0.45/hr.
- Short, discrete clips: the Sync API. One HTTP POST returns a finished transcript in about 134 ms at the median, at $0.30/hr.
- High-volume streaming where flagship accuracy isn't required: Universal-Streaming at $0.15/hr for English and six-language multilingual workloads.
- Long-form or meeting-heavy audio: test carefully. The benchmarks show Universal-3.6 Pro Realtime trailing on long-form English and diarization, so run your own audio through each finalist.
- Self-hosted deployment: WhisperX or Whisper Streaming, if you have the ML engineering capacity to run them.
Proof-of-concept checklist
Before committing, test with your own use case:
- Run representative audio that matches your real conditions, including noise and accents.
- Measure entity accuracy on the names, numbers and IDs your product depends on.
- Hold sessions open for full call lengths and measure what you're billed for idle time.
- Test new-session rate limits and reconnect handling at expected peak load.
- Price it against projected usage, including add-ons.
Final words
Real-time speech recognition in 2026 is no longer about whether a model can keep up. It's about whether it understands the conversation well enough to get the name, the number and the turn end right, and whether the connection behaves predictably at scale. That's where the gap between streaming APIs is widest.
No model wins every category, including Universal-3.6 Pro Realtime. So use the ranks above to build a shortlist, then test the finalists on your own audio before you commit.
Get a free API key and stream your first transcript in minutes, with keyterms included and no spend commitment to scale.
Frequently Asked Questions (FAQs)
Which real-time speech-to-text API is most accurate?
On AssemblyAI's published benchmarks, Universal-3.6 Pro Realtime has the lowest voice-agent entity error rate of 22 real-time systems at 14.41%, ahead of Soniox Realtime at 18.19%. It also ranks 1st on English short-form WER and code-switching. It trails on long-form English (6th of 22) and diarization (3rd of 12), so test it on audio like yours.
Which streaming speech-to-text API handles noisy audio best?
Universal-3.6 Pro Realtime ranks 1st of 22 on voice-agent WER in heavy natural noise, at 6.13% against 7.68% for Soniox Realtime. Its Voice Focus add-on (+$0.10/hr) isolates the primary speaker in near-field or far-field audio. Background voices are the hardest case, so test with recordings from your real environment.
How is real-time speech-to-text billed?
Most streaming APIs bill per hour or per minute, but what's metered differs. AssemblyAI bills streaming on how long the WebSocket session stays open, at $0.45/hr for Universal-3.6 Pro Realtime and $0.15/hr for Universal-Streaming. Close sessions when calls end, because idle connection time counts.
Which real-time speech-to-text APIs support speaker diarization?
AssemblyAI, Azure, Speechmatics, Meta, Reson8 and xAI are among the systems with streaming diarization on the /benchmarks diarization table. Universal-3.6 Pro Realtime posts a 16.68% diarization error rate, 3rd of 12, with mid-session speaker revisions at +$0.12/hr. Meta Muse Voice Transcribe leads that table at 14.51%.
Is Google or Azure real-time speech-to-text accurate enough for production voice agents?
On AssemblyAI's voice-agent benchmark, Azure posts 29.7% entity error and Google Gemini 3.5 Live 20.1%, against 14.4% for Universal-3.6 Pro Realtime. That means nearly one in three spoken entities comes back wrong on Azure in that test. If your agent acts on names, numbers or account IDs, measure entity accuracy on your own calls before you commit to either.