Benchmarks

AssemblyAI Universal-3.6 Pro Realtime benchmarks

Universal-3.6 Pro Realtime measured against every major streaming vendor on the same audio.

Pre-recorded speech recognition, measured against the field

Universal-3.5 Pro and models from 11 speech-to-text providers across languages, conditions, and tasks: same audio, same scoring pipeline.

Model under test Universal-3.5 Pro (production endpoint) Evaluation AssemblyAI Research Rivals run via each vendor’s public API, default multilingual settings Snapshot 2026-10-14

01English

Word error rate (WER) is the most widely used metric for measuring the accuracy of speech-to-text models. It is the percentage of words that a speech recognition model gets wrong, calculated by comparing the model’s output with a human-generated transcript. Lower is better.

Clean transcription

Human transcripts vary depending on transcription style, which can broadly be classified as either clean or verbatim. Clean transcripts omit non-content speech elements, such as stutters, repetitions, and false starts, to improve readability, whereas verbatim transcripts capture all of these elements. Because WER depends on the reference transcript, we present results for each transcription style.

Figure 1 Average word error rate, clean-reference datasets (lower is better)

Table 1 Per-dataset WER, clean-reference datasets (%, lower is better) English datasets with clean-style reference transcripts · normalized word error rate · every vendor's production pre-recorded API, default settings · September 2026. · Mean is the unweighted average of the columns shown.

Per-dataset WER, clean-reference datasets (%, lower is better)
System	Mean	Common Voice	LibriSpeech (clean)	LibriSpeech (other)	Meanwhile	Broadcast News	General Audio	Medical Dialogue (Specialist)	Noisy Speech	Spoken Numbers	Webinars
AssemblyAI Universal-3.5 Pro	5.0	5.4	1.6	2.7	4.2	4.5	8.7	1.0	10.9	4.9	6.0
Cohere Transcribe (language pinned)	5.0	6.2	1.3	2.4	3.9	4.3	8.9	1.8	10.6	5.0	5.9
Voxtral Mini Transcribe 2	5.3	6.7	1.6	3.3	3.9	4.3	8.3	1.9	11.1	5.0	6.7
OpenAI GPT-4o Transcribe	5.5	8.3	1.5	3.9	3.8	4.4	9.1	1.0	12.8	4.0	6.5
Speechmatics Enhanced	6.2	6.9	2.2	4.8	4.5	5.0	9.6	1.0	11.9	8.4	8.1
Soniox stt-async-v5	6.3	13.8	2.8	6.4	4.4	4.6	8.7	1.0	11.4	3.6	6.5
ElevenLabs Scribe v2	6.4	5.1	1.5	2.8	2.9	6.5	11.2	0.8	14.3	9.4	9.9
Gemini 3.5 Transcribe	7.0	8.0	2.1	4.7	3.3	5.1	11.9	0.9	18.3	6.8	9.3
AWS Transcribe	7.4	5.7	1.7	3.3	5.0	5.3	10.8	1.4	24.0	6.9	9.9
Parakeet TDT 0.6B v3	8.1	7.3	1.9	3.6	4.9	9.7	12.2	4.5	15.2	10.3	11.6
Deepgram Nova-3	8.4	19.4	2.8	6.5	5.3	5.8	10.2	1.2	14.1	8.8	9.5
Azure Batch Transcription	8.4	15.5	2.4	5.2	7.0	6.1	11.4	1.7	14.5	9.7	10.4
Gladia Solaria-1	9.5	10.9	1.9	4.0	10.1	5.0	10.7	1.4	40.0	4.5	6.8
Grok STT	9.8	6.6	1.8	3.8	5.3	9.3	14.3	1.7	17.4	27.0	10.6

Best Worst Cells are coloured by rank within each column (solid blue is best, white is worst).

Verbatim transcription

† Mean over the datasets with results. The vendor's API failed repeatedly on the remaining datasets, so those cells are blank and excluded from the mean.

Figure 2 Average word error rate, verbatim-reference datasets (lower is better)

Table 2 Per-dataset WER, verbatim-reference datasets (%, lower is better) English datasets with verbatim-style reference transcripts · normalized word error rate · every vendor's production pre-recorded API, default settings · September 2026. · Mean is the unweighted average of the columns shown. · † Mean over the datasets with results. The vendor's API failed repeatedly on the remaining datasets, so those cells are blank and excluded from the mean.

Per-dataset WER, verbatim-reference datasets (%, lower is better)
System	Mean	AMI (headset)	CORAAL	CORAAL (long-form)	Earnings-21	Earnings-22	Earnings-22 (full)	PriMock57	Rev16	Alphanumeric Codes	Broadcast News	General Audio	Noisy Speech	Numbers in Conversation	Respiratory Consultations	Spoken Email Addresses	Verbatim Disfluent Speech	Webinars
ElevenLabs Scribe v2	7.2	10.1	7.0	10.8	6.5	7.1	10.2	10.4	10.5	3.5	3.5	6.4	9.0	6.0	10.4	4.6	2.4	3.5
AssemblyAI Universal-3.5 Pro	8.0	12.8	10.2	14.5	8.7	6.5	12.7	11.0	9.9	3.7	3.5	5.4	6.3	4.3	8.8	5.3	7.0	5.1
AWS Transcribe	8.6	10.0	8.6	15.0	7.5	9.6	11.3	7.9	10.5	4.5	4.8	7.4	19.9	6.5	7.1	8.1	3.5	4.3
Soniox stt-async-v5	9.1	22.5	11.4	13.2	9.8	10.4	13.0	12.1	10.5	4.1	4.2	6.8	7.5	4.1	9.5	6.2	5.0	4.8
Parakeet TDT 0.6B v3	9.4	10.6	8.3	15.1	9.8	11.9	14.2	12.9	11.1	5.2	7.6	8.5	10.0	6.4	10.0	8.6	2.9	7.0
Azure Batch Transcription	9.7	16.1	11.9	17.8	8.0	11.7	12.2	9.1	11.2	6.1	4.7	8.2	9.4	6.9	7.0	10.9	7.8	5.3
Voxtral Mini Transcribe 2	9.7	13.7	12.7	18.5	9.8	10.3	13.4	14.2	10.7	4.3	4.8	6.9	9.1	4.8	10.4	6.8	8.5	5.6
Deepgram Nova-3	9.8	17.1	10.8	17.9	11.2	12.0	14.5	11.5	10.7	5.5	4.2	7.1	10.0	7.3	9.4	6.3	6.4	4.3
Cohere Transcribe (language pinned)	9.8	9.0	12.3	17.6	10.2	9.8	13.9	20.8	11.4	3.9	4.8	7.5	9.1	4.8	10.4	7.4	7.7	6.0
Gladia Solaria-1	12.3	18.4	13.1	21.6	10.1	10.9	15.3	13.7	10.5	4.2	5.3	9.1	38.6	5.9	10.1	7.6	8.0	6.2
OpenAI GPT-4o Transcribe	13.1	36.9	34.4	20.0	10.2	10.1	14.7	14.3	12.4	4.3	5.1	8.3	11.4	4.4	11.2	6.4	10.4	7.6
Speechmatics Enhanced	8.5	—	10.4	14.5	7.8	10.7	12.4	11.0	9.8	4.8	4.2	6.4	8.2	6.0	9.0	9.4	6.4	4.8
Gemini 3.5 Transcribe	8.6	11.6	9.4	14.7	9.8	5.1	22.6	10.6	—	3.2	3.8	8.6	10.7	5.1	9.1	4.5	5.1	3.8
Grok STT	10.4	9.9	9.7	17.4	9.3	6.5	12.0	12.3	—	9.3	7.3	10.5	11.6	22.1	9.7	9.5	4.8	4.8

Best Worst Cells are coloured by rank within each column (solid blue is best, white is worst).

Hear it for yourself

Five recordings from the English sets, each transcribed by the models in the tables above. The first three rows are fixed; pick any other model in the fourth.

1 of 5

Common Voice

0:00 / 0:00

AssemblyAI Universal-3.5 Pro

Among these people were a couple of cyclists, a gardener I employ sometimes, and a girl carrying a baby.

ElevenLabs Scribe v2

Among these people were a couple of cl-cy-cyclists, a gardener, a employee, some peasants, and a girl carrying a baby

Deepgram Nova-3

Among these people were couple of car car cyclists. Gardener, I employed some titan and a girl carrying a baby.

Mount this paper with a couple of cyclists, the gardener I employ sometimes and a girl carrying a baby.

Accented speech

A good speech-to-text model delivers accurate transcriptions across accents and dialects.

† Mean over the datasets with results. The vendor's API failed repeatedly on the remaining datasets, so those cells are blank and excluded from the mean.

Figure 3 Average word error rate for accented speech recognition (lower is better)

Table 3 Per-dataset accented-speech WER (%, lower is better) Accented English datasets (read speech from British, French-L1, Indian and Spanish-L1 speakers) · normalized word error rate · every vendor's production pre-recorded API, default settings · September 2026. · Mean is the unweighted average of the columns shown. · † Mean over the datasets with results. The vendor's API failed repeatedly on the remaining datasets, so those cells are blank and excluded from the mean.

Per-dataset accented-speech WER (%, lower is better)
System	Mean	British	French	Indian	Spanish
ElevenLabs Scribe v2	7.0	5.5	8.0	5.8	8.9
AssemblyAI Universal-3.5 Pro	7.1	5.3	8.3	5.3	9.4
Voxtral Mini Transcribe 2	8.7	5.8	10.6	6.5	11.8
Gemini 3.5 Transcribe	9.2	6.3	11.3	6.5	12.6
Grok STT	9.4	6.8	11.6	6.6	12.7
Azure Batch Transcription	9.6	6.9	10.4	10.7	10.2
Cohere Transcribe (language pinned)	10.1	7.6	12.8	6.4	13.5
AWS Transcribe	10.8	6.5	15.4	5.9	15.1
OpenAI GPT-4o Transcribe	11.8	5.7	11.8	16.8	12.8
Parakeet TDT 0.6B v3	13.4	7.7	18.0	6.9	21.2
Soniox stt-async-v5	13.5	8.2	13.3	17.4	14.9
Gladia Solaria-1	13.7	7.8	21.8	11.3	13.7
Deepgram Nova-3	14.4	9.3	20.3	8.2	19.8
Speechmatics Enhanced	7.5	6.7	—	8.2	—

Best Worst Cells are coloured by rank within each column (solid blue is best, white is worst).

02Named Entities

Named entities refer to the names of people, organizations, and products; locations; dates and times; quantities; email addresses; phone numbers; and so on - unique real-world objects or concepts. They carry the most semantic weight in downstream applications, and accurately capturing them makes a real difference. Missed entity rate is the percentage of entity words that a speech recognition model misses. Lower is better.

Table 4 Missed-entity error rate by entity type (%, lower is better) An 18,700-file named-entity benchmark (~57,000 entities across 11 categories) · missed-entity error rate, text-normalized per entity category · every vendor's production pre-recorded API, default settings · September 2026. · Mean is the unweighted average of the columns shown.

Missed-entity error rate by entity type (%, lower is better)
System	Mean	Names	Organizations	Locations	Occupations	Medical terms	Dates & times	Monetary amounts	Phone numbers	Credit card numbers	Email addresses	URLs
Gemini 3.5 Transcribe	14.5	14.9	17.3	8.3	6.3	10.4	4.4	20.7	10.2	5.3	31.7	30.2
AssemblyAI Universal-3.5 Pro	15.2	13.1	12.6	6.5	5.4	9.7	3.5	26.3	13.4	15.3	32.7	29.1
Soniox stt-async-v5	16.3	14.1	19.2	8.4	6.5	11.4	5.1	24.0	12.9	15.9	33.5	28.6
AWS Transcribe	19.7	16.6	23.1	7.7	6.2	14.6	4.7	18.2	14.4	11.8	70.6	29.1
Voxtral Mini Transcribe 2	21.5	16.3	15.8	8.0	6.6	15.2	3.6	23.7	14.0	16.2	73.1	43.7
Cohere Transcribe (language pinned)	22.3	14.8	18.6	7.2	6.5	16.1	4.2	22.0	14.7	21.2	73.9	46.1
Gladia Solaria-1	22.4	14.4	18.1	7.5	6.4	14.5	4.0	24.6	17.3	24.5	73.1	42.2
Deepgram Nova-3	22.5	15.0	14.7	11.3	6.3	12.7	10.1	69.6	17.3	15.6	40.9	34.1
OpenAI GPT-4o Transcribe	22.9	19.2	18.2	10.7	9.9	11.9	7.8	25.4	12.9	17.7	74.1	43.7
Speechmatics Enhanced	22.9	16.8	21.3	9.2	6.4	11.7	4.2	20.2	19.6	20.1	75.1	47.1
Azure Batch Transcription	22.9	17.9	22.8	17.0	7.5	17.1	4.8	24.5	13.1	20.6	63.5	43.7
ElevenLabs Scribe v2	24.7	11.3	12.9	10.1	5.7	8.7	13.0	66.2	47.2	42.2	29.2	25.2
Parakeet TDT 0.6B v3	27.6	17.3	24.1	9.6	8.2	21.6	10.7	30.4	20.6	23.9	79.2	57.8
Grok STT	44.8	17.7	20.9	10.3	7.3	17.1	15.5	66.7	82.3	87.3	86.6	80.8

Best Worst Cells are coloured by rank within each column (solid blue is best, white is worst).

We define a named entity as correct when it matches how a human would normally write it, ignoring spelling ambiguities and non-semantic differences. As a result, the metric is still somewhat affected by transcription-style differences that survive normalization.

03Diarization

Speaker diarization tags transcript segments by speaker. Although the commonly used diarization error rate (DER) is convenient, it is highly sensitive to segmentation policies that are irrelevant to user-perceived quality. Therefore, we first report concatenated minimum-permutation word error rate (cpWER). It jointly measures transcription and speaker-assignment accuracy by finding the best match between predicted and reference speakers, and it is insensitive to differences in segmentation policies. Our post on DER versus cpWER walks through the difference in detail. For completeness, we also report the conventional DER. Lower is better.

Figure 4 Average concatenated minimum-permutation word error rate (lower is better)

Table 5 Per-dataset cpWER (%, lower is better) Every vendor’s production pre-recorded API, default settings · October 2026.

Per-dataset cpWER (%, lower is better)
System	Mean	AMI	CallHome	DiPCo	NOTSOFAR
ElevenLabs Scribe v2	26.4	25.4	14.2	34.3	31.7
AssemblyAI Universal-3.5 Pro	26.7	26.2	17.6	29.1	33.7
AWS Transcribe	32.0	25.9	23.8	35.5	42.9
Speechmatics Enhanced	33.7	20.7	21.8	35.5	56.6
Deepgram Nova-3	34.8	29.5	24.4	37.2	48.2
Grok STT	36.0	32.3	24.3	43.4	44.1
Gladia Solaria-1	36.5	34.5	20.1	43.5	47.9
Voxtral Mini Transcribe 2	38.1	34.6	26.6	39.3	52.0
Soniox stt-async-v5	43.8	59.8	17.2	42.4	55.9

Best Worst Cells are coloured by rank within each column (solid blue is best, white is worst).

Figure 5 Average diarization error rate (lower is better)

Table 6 Per-dataset DER (%, lower is better) DER was measured on both the datasets shared with the cpWER evaluation and additional datasets that provide only speaker-segment labels · every vendor’s production pre-recorded API, default settings · October 2026.

Per-dataset DER (%, lower is better)
System	Mean	AMI	CallHome	DiPCo	NOTSOFAR	VoxConverse	MSDWild
AssemblyAI Universal-3.5 Pro	12.2	19.4	14.7	12.4	5.3	10.2	11.2
ElevenLabs Scribe v2	14.6	25.6	11.6	17.3	10.9	11.5	10.6
Grok STT	15.4	21.5	17.4	20.2	8.9	10.7	13.9
Voxtral Mini Transcribe 2	15.9	22.4	13.9	15.4	17.3	11.4	15.2
Speechmatics Enhanced	16.2	16.9	15.0	14.3	22.4	10.7	17.9
Gladia Solaria-1	16.7	23.1	15.4	20.8	13.5	10.5	16.8
AWS Transcribe	16.9	22.1	20.2	17.1	12.6	11.2	18.1
Deepgram Nova-3	17.4	19.0	18.2	16.4	14.2	11.9	24.6
Soniox stt-async-v5	24.2	46.4	14.5	22.8	26.6	19.6	15.3

Best Worst Cells are coloured by rank within each column (solid blue is best, white is worst).

04Code switching

Code-switching, switching between two or more languages within a conversation or utterance, is measured by WER using reference transcripts.

† Mean over the datasets with results. The vendor's API failed repeatedly on the remaining datasets, so those cells are blank and excluded from the mean.

Figure 6 Average word error rate for code-switching speech recognition (lower is better)

Table 7 Per-dataset code-switching WER (%, lower is better) English paired with German, Spanish, French, Italian and Portuguese · normalized word error rate · every vendor’s production pre-recorded API, default settings · September 2026. · Mean is the unweighted average of the columns shown. · † Mean over the datasets with results. The vendor's API failed repeatedly on the remaining datasets, so those cells are blank and excluded from the mean.

Per-dataset code-switching WER (%, lower is better)
System	Mean	Long EN-DE	Long EN-ES	Long EN-FR	Long EN-IT	Long EN-PT	Short EN-DE	Short EN-ES	Short EN-FR	Short EN-IT	Short EN-PT	Short EN-DE (cleaner)	Short EN-ES (cleaner)	Short EN-FR (cleaner)	Short EN-IT (cleaner)	Short EN-PT (cleaner)	Conversational EN-ES
AssemblyAI Universal-3.5 Pro	7.7	7.3	6.1	8.6	7.2	8.1	7.0	6.5	8.8	6.7	11.3	4.9	5.2	5.4	5.3	5.5	19.4
ElevenLabs Scribe v2	8.6	6.6	5.8	7.9	6.0	6.2	9.2	7.2	9.6	9.6	10.0	4.7	5.2	9.0	7.4	6.9	26.7
Soniox stt-async-v5	10.2	9.3	8.4	11.1	8.7	9.3	11.3	10.3	14.2	10.7	12.9	5.9	6.4	7.0	6.3	6.0	24.6
Azure Batch Transcription	17.4	16.1	16.1	19.0	14.4	17.3	17.4	19.4	20.7	18.9	20.1	10.7	11.4	13.1	11.9	13.1	38.1
Voxtral Mini Transcribe 2	17.4	11.2	10.5	14.5	11.5	11.5	27.4	24.4	29.9	27.6	22.2	11.3	8.1	18.0	12.6	8.8	28.8
AWS Transcribe	22.9	21.5	16.7	23.0	17.9	34.8	21.4	20.5	25.1	18.2	26.0	22.1	17.5	24.8	17.0	20.1	39.7
Parakeet TDT 0.6B v3	29.3	63.2	54.1	70.1	28.8	40.4	13.0	14.3	17.7	17.1	16.1	15.2	20.1	16.7	15.8	25.1	41.0
Cohere Transcribe (language pinned)	29.4	28.3	33.0	37.7	28.5	21.6	27.4	38.6	32.5	27.3	24.6	18.6	43.8	26.1	13.7	18.5	50.9
OpenAI GPT-4o Transcribe	44.7	31.4	33.0	37.4	31.7	33.5	52.6	50.5	51.7	50.0	54.4	52.5	48.0	49.9	48.4	51.1	38.9
Deepgram Nova-3	49.4	54.0	43.2	54.3	49.5	51.0	50.9	47.9	51.9	48.1	48.0	49.3	43.3	53.4	50.3	49.9	46.1
Gladia Solaria-1	50.4	42.3	43.3	43.7	45.3	40.0	46.4	43.5	47.4	43.6	44.1	57.5	49.5	53.5	58.5	57.6	90.7
Speechmatics Enhanced	26.3	22.8	—	29.8	—	—	—	—	—	—	—	—	—	—	—	—	—
Gemini 3.5 Transcribe	36.3	40.0	40.8	29.6	32.8	41.9	34.9	35.3	38.8	37.0	44.8	33.9	34.4	38.3	29.3	32.8	—
Grok STT	37.8	30.6	28.4	32.5	27.9	34.5	46.7	43.6	47.2	42.7	48.0	—	—	—	—	—	34.3

Best Worst Cells are coloured by rank within each column (solid blue is best, white is worst).

05Multilingual

Models are evaluated without prior knowledge of the language spoken in each audio file. Therefore, they perform both language detection and speech recognition, reflecting the most common multilingual use case, in which the spoken language is not known in advance. Azure and Cohere were run with the target language specified.

† Mean over the datasets with results. The vendor's API failed repeatedly on the remaining datasets, so those cells are blank and excluded from the mean.

Figure 7 Average word error rate across nine languages (lower is better)

Table 8 Per-language error rate (lower is better) Each language is the unweighted average of its Common Voice and FLEURS sets; Mean is the unweighted average of the nine languages. Japanese and Chinese are scored by character error rate. · † One of the language's two sets was gated, so the cell is its one available set; a language with both sets gated is blank and out of the Mean.

Per-language error rate (lower is better)
System	Mean	Spanish	French	German	Italian	Portuguese	Dutch	Hindi	Japanese · CER	Chinese · CER
ElevenLabs Scribe v2	6.7	3.5	5.2	4.4	3.0	5.0	3.8	18.9	9.3	7.7
AWS Transcribe	6.9	4.3	7.5	5.2	4.2	6.5	4.8	6.4	10.9	12.4
Voxtral Mini Transcribe 2	7.2	3.8	6.5	4.8	4.0	6.0	5.0	10.8	10.9	13.0
AssemblyAI Universal-3.5 Pro	7.2	3.9	6.3	4.8	3.8	6.9	4.3	12.9	12.1	10.2
Gemini 3.5 Transcribe	7.8	4.3	7.8	5.8	4.7	7.9	5.9	8.3	13.7	11.6
Soniox stt-async-v5	8.7	5.0	8.9	6.2	5.0	6.9	5.3	16.9	12.4	11.3
OpenAI GPT-4o Transcribe	8.7	3.7	6.7	4.4	3.7	6.2	5.4	24.4	12.1	11.4
Azure Batch Transcription	10.4	8.8	11.1	8.1	8.2	10.0	10.0	8.9	17.1	11.0
Gladia Solaria-1	11.6	4.8	9.4	6.3	5.4	6.9	6.5	32.5	13.6	18.7
Grok STT	14.0	5.1	8.0	8.1	5.5	9.3	8.8	13.0	17.9	49.9
Parakeet TDT 0.6B v3	5.9	4.5	—	5.9	4.5	7.3	7.0	—	—	—
Cohere Transcribe (language pinned)	6.8	4.0	6.0	5.7	3.9	5.3	5.5	—	11.4	12.5
Speechmatics Enhanced	14.3	4.3	7.6	6.4	5.4	7.3	11.7	—	—	57.2
Deepgram Nova-3	22.4	10.9	12.5	10.4	8.1	13.8	7.9	28.6	86.6	—

Best Worst Cells are coloured by rank within each column (solid blue is best, white is worst).

06How we measure

Every number on this page comes from the same protocol: one production run per system on the same audio, one normalizer, one metric per benchmark. This section records exactly how the runs were made, which systems and datasets they cover, and how each benchmark is scored.

How the runs were made

Protocol. Each system is run once through its public pre-recorded API with default settings, one model per vendor, on the same audio. Language is left to the vendor’s detection or default; Cohere is fixed to English and Azure is sent an English-first candidate list.

Complete runs. A cell is shown only when the system returned results for at least 90% of the files our own run scored on that dataset; otherwise it is blank and left out of the mean. Rows with a blank cell are listed after complete rows and marked with a dagger.

Excluded. Vendors whose terms prohibit competitive benchmarking, unreleased models, and superseded runs of the same model.

Snapshot. Results are a snapshot of runs completed by 2026-10-14.

Systems

The vendors and models evaluated, the language setting and run dates for each, and the benchmarks each has a current run on.

Table 9 Systems evaluated and the benchmarks each has a current run on. Model identifiers and language settings taken from each vendor’s source-job configuration · September 2026.

Systems evaluated and the benchmarks each has a current run on.
Vendor	Model	Language	English	Entities	Code-switching	Diarization	Multilingual	Runs
AssemblyAI	universal-3-5-pro	Auto-detect	●	●	●	●	●	2026-07-27 → 2026-10-13
Deepgram	nova-3-general	Auto-detect	●	●	●	●	●	2026-08-20 → 2026-10-14
ElevenLabs	scribe_v2	Auto-detect	●	●	●	●	●	2026-08-20 → 2026-10-14
Speechmatics	enhanced	Auto-detect	●	●	●	●	●	2026-08-20 → 2026-10-14
OpenAI	gpt-4o-transcribe	Auto-detect	●	●	●	·	●	2026-08-20 → 2026-09-14
Google	gemini-3.5-transcribe-preview	Auto-detect	●	●	●	·	●	2026-09-08 → 2026-09-14
AWS	Transcribe	Auto-detect	●	●	●	●	●	2026-08-21 → 2026-10-14
Microsoft	Batch Transcription	Target locale as primary, with language identification	●	●	●	·	●	2026-08-21 → 2026-09-13
Gladia	pre-recorded API	Auto-detect	●	●	●	●	●	2026-08-20 → 2026-10-14
Soniox	stt-async-v5	Auto-detect	●	●	●	●	●	2026-08-20 → 2026-10-14
Mistral	voxtral-mini-latest	Auto-detect	●	●	●	●	●	2026-08-20 → 2026-10-14
xAI	stt	Auto-detect	●	●	●	●	●	2026-09-06 → 2026-10-14
NVIDIA	nvidia/parakeet-tdt-0.6b-v3	Auto-detect	●	●	●	·	●	2026-08-20 → 2026-08-29
Cohere	cohere-transcribe-03-2026	Target language specified	●	●	●	·	●	2026-04-15 → 2026-09-13

Test data

The datasets behind each benchmark, the number of audio files scored, and the metric.

Table 10 Datasets, audio-file counts and metric per benchmark.

Datasets, audio-file counts and metric per benchmark.
Benchmark	Audio	Files	Scored on
Clean transcription	Common Voice (Crowdsourced read sentences from volunteer speakers worldwide) · LibriSpeech test-clean (Read audiobook speech recorded in clean, low-noise conditions) · LibriSpeech test-other (Read audiobook speech in more challenging acoustic conditions) · Meanwhile (Late-night monologue segments with fast, idiomatic speech) · Television and radio news broadcasts with professional narration. · General-purpose real-world audio across mixed recording conditions. · Specialist medical dialogue with clinical terminology. · Speech recorded with heavy real-world background noise. · Conversational audio dense with spoken numeric sequences. · Recorded webinar and online-presentation audio.	21,808	WER
Verbatim transcription	AMI headset (Multi-party meeting recordings captured on individual headset microphones) · CORAAL (Interview recordings of regional African American English) · CORAAL long-form (Long-form interview recordings of regional African American English) · Earnings-21 (Corporate earnings calls from 2021, dense with financial terminology) · Earnings-22 (Corporate earnings calls from 2022, dense with financial terminology) · Earnings-22 full (Full 2022 corporate earnings-call set) · PriMock57 (Simulated primary-care consultations between clinicians and patients) · Rev16 (Podcast episodes spanning varied speakers and topics) · Spoken alphanumeric codes such as IDs and reference numbers. · Television and radio news broadcasts with professional narration. · General-purpose real-world audio across mixed recording conditions. · Speech recorded with heavy real-world background noise. · Scripted call-center conversations dense with spoken numbers. · Respiratory-care consultations with clinical terminology. · Email addresses dictated aloud in conversation. · Spontaneous English speech with verbatim references that keep fillers, repeats and false starts · Recorded webinar and online-presentation audio.	15,300	WER
Accented speech	English speech from British-accented speakers. · English speech from native French speakers. · English speech from Indian-accented speakers. · English speech from native Spanish speakers.	9,292	WER
Entities	A named-entity benchmark spanning 11 entity categories: Names, Organizations, Locations, Occupations, Medical terms, Dates & times, Monetary amounts, Phone numbers, Credit card numbers, Email addresses, and URLs.	18,700	Missed entity rate
Code-switching	Synthetic and conversational bilingual recordings for five language pairs: English–German, English–Spanish, English–French, English–Italian, English–Portuguese.	9,090	WER
Diarization	Multi-party meeting recordings. · Telephone conversations. · Dinner-party conversations. · Meeting recordings from the NOTSOFAR challenge. · In-the-wild multi-speaker media audio. · Multi-speaker conversational video audio.	1,306	cpWER and DER
Multilingual	FLEURS and Common Voice test sets in nine languages: Spanish, French, German, Italian, Portuguese, Dutch, Hindi, Japanese, Chinese.	110,630	WER; CER for Japanese and Chinese

How we score

One normalizer and one metric per benchmark, applied the same way to every system.

WER. Word error rate: the percentage of words a system gets wrong against a human reference. Lower is better.

Missed entity rate. The share of entity words missed after normalization, per entity category.

cpWER. Concatenated minimum-permutation word error rate: WER computed under the best match between predicted and reference speakers, so it scores transcription and speaker assignment together.

DER. Diarization error rate: the share of speaking time attributed to the wrong speaker — missed speech, false alarm and speaker confusion. DER uses a 0.25-second collar around reference boundaries and does not score overlapping speech.

Mean. The unweighted average of the columns shown.

Normalization. One normalizer (Whisper’s text normalizer: case, punctuation, numbers, abbreviations) is applied to references and every system’s output before scoring.

Entity scoring: in addition to the Whisper text normalizer, we applied custom normalization steps that accept equivalent spellings of ambiguous proper nouns and ignore non-semantic differences.

Blank cells. A cell is left blank when a system returned no usable transcript for more than 10% of a dataset’s files; those files are excluded rather than scored, and the row is listed after the complete rows. Speechmatics, run with its automatic language identification, rejected a fixed subset of short and bilingual recordings as having no identifiable language. Grok and Gemini 3.5 Transcribe returned empty transcripts for some recordings, Grok on short bilingual clips and Gemini on long recordings.

Benchmarks

Run your own benchmark

Testing on your own audio? Want to do it right? Read our benchmarking guide in the docs.

Learn more Get a custom benchmark

Technical report Realtime speech-to-text data as of 2026-09-16 v2026.09

Realtime speech recognition, measured against the field

Universal-3.6 Pro Realtime and 26 competitor models on 5 evaluation suites: same audio, same scoring pipeline.

Model under test Universal-3.6 Pro Realtime (production endpoint) Evaluation AssemblyAI Research Rivals run via each vendor’s public streaming API, default settings Snapshot 2026-09-16

Universal-3.6 Pro Realtime has the lowest entity error rate on the voice-agent suite: 14.41% against 18.19% for the nearest competing system, a 21% lead. It also leads on code-switched speech.

Every score is in the tables. §9 reproduces independent evaluations; §10 describes the method.

At a glance

Voice-agent entity error

14.41%

1st of 22

next best Soniox Realtime 18.19%

Code-switching (5 pairs)

7.20%

1st of 23

next best AWS Transcribe 9.13%

Voice-agent WER, heavy noise

6.13%

1st of 22

next best Soniox Realtime 7.68%

English short-form WER

3.44%

1st of 22

next best Reson8 3.70%

Voice-agent WER, Indian-accented

6.64%

1st of 22

next best Soniox Realtime 7.78%

01Overview

Each cell is the vendor’s best model on that metric, coloured by rank within the column (solid blue is first). Values are in the tables of §2–§8.

Rank across 9 headline metrics, vendors ordered by mean rank.
Vendor (best model per cell)	VA entity err. NEER %	Entity (judge) %	Medical MER %	Diarization DER %	Non-speech resp. %	Code-switching WER %	Multi-lingual FLEURS %	EN short-form WER %	EN long-form WER %	meanrank
AssemblyAI	14.4	18.1	4.2	16.0	0.3	7.2	8.4	3.4	8.8	3.0
Google	20.1	19.7	8.6	—	0.2	10.2	6.4	5.0	9.4	5.3
Meta	22.2	20.5	4.1	14.5	0.3	13.3	10.0	6.2	7.4	5.9
Reson8	23.7	24.4	14.3	16.7	0.7	10.7	5.8	3.7	7.7	6.0
Speechmatics	20.6	19.8	4.7	22.2	0.2	38.2	11.1	4.7	8.1	6.6
Soniox	18.2	21.5	4.6	31.8	0.6	11.3	7.9	7.6	8.2	6.7
AWS	19.3	28.1	10.2	27.5	1.3	9.1	8.9	3.9	8.7	7.2
ElevenLabs	18.5	20.4	7.0	—	0.2	13.0	9.3	5.1	17.1	7.4
Zoom	18.2	24.9	10.5	—	0.3	32.5	6.2	3.8	9.3	7.9
Mistral	23.1	20.8	12.3	—	0.0	18.5	9.5	6.6	18.9	10.0
Azure	29.7	33.1	14.4	20.4	0.2	14.6	12.7	5.5	9.2	10.2
Deepgram	26.1	25.9	13.2	34.0	0.2	17.1	12.7	6.5	9.6	10.4
xAI	24.0	26.1	15.7	25.7	1.2	20.6	14.8	5.8	9.4	11.6
OpenAI	38.1	20.2	17.2	—	0.1	33.3	12.3	8.0	15.9	12.5
Smallest	29.4	33.2	21.4	41.4	1.4	14.6	11.9	5.4	10.9	12.6
Gladia	30.6	24.9	17.0	—	4.9	17.1	16.2	7.0	10.7	14.1
Cartesia	40.1	27.8	21.1	—	3.1	50.5	18.7	7.7	10.6	16.1

Best Worst Cells are coloured by rank within each column (solid blue is best, white is worst).

Figure 1 Rank across 9 headline metrics, vendors ordered by mean rank. source = the tables below; each column’s provenance is given with its table

Position in the field

Voice-agent entity error 14.41% · 1st of 22 systems · nearest competitor Soniox Realtime 18.19% −20.8%
Judge-scored entity error 18.93% · 2nd of 22 systems · nearest competitor Google Gemini 3.5 Live 19.68% −3.8%
Medical entity error 4.24% · 2nd of 22 systems · nearest competitor Meta Muse Voice Transcribe 4.08% +3.9%
Diarization DER 16.68% · 3rd of 12 systems · nearest competitor Meta Muse Voice Transcribe 14.51% +15.0%
Non-speech response rate 0.27% · 9th of 22 systems · nearest competitor Mistral Voxtral Mini 0.03% +0.2 pt
Code-switching (5 pairs) 7.20% · 1st of 23 systems · nearest competitor AWS Transcribe 9.13% −21.1%
Multilingual (FLEURS) 8.44% over 18 lang. · 5th of 22 systems · nearest competitor Reson8 5.77% over 7 lang. +46.3%
English short-form WER 3.44% · 1st of 22 systems · nearest competitor Reson8 3.70% −7.0%
Long-form English WER 8.82% · 6th of 22 systems · nearest competitor Meta Muse Voice Transcribe 7.37% +19.7%

We lead We trail The margin is blue where we lead, grey where we trail.

One row per headline metric from the grid above: our value, our rank among systems with a complete run, and the margin to the runner-up where we lead or to the leader where we do not. Margins are relative for error rates and absolute for rates below one percent.

02Voice agents

Conversational audio as a voice agent hears it, across 22 systems, three noise environments, 12 speaker groups and three utterance lengths. The headline metric, entity error (NEER), is how often the names, addresses and numbers an agent acts on come out wrong.

Table 1 Entity error rate (%) by entity class. Colour is rank within the column. We lead in 9 of 11 columns. normalized entity error rate (NEER) · runs 2026-08-27 → 2026-09-22 · one row per model, most recent run · corpus value = mean of the three natural-noise environments

Entity error rate (%) by entity class. Colour is rank within the column. We lead in 9 of 11 columns.
System	Overall	Names	Places	Addresses	Phone	Email	Dates	Numbers	Codes/IDs	Medical	Technical
AssemblyAI Universal-3.6 Pro	14.4	10.9	6.0	29.0	2.4	17.2	18.6	12.9	10.0	16.1	20.9
AssemblyAI Universal-3.5 Pro	14.8	11.2	7.3	27.9	2.6	17.1	19.1	13.2	10.2	17.3	21.7
Soniox Realtime	18.2	16.6	7.7	27.9	2.7	22.7	30.8	19.7	8.2	19.1	26.5
Zoom Scribe (Live)	18.2	16.5	7.0	30.0	9.6	22.4	20.9	14.9	13.9	20.0	26.4
ElevenLabs Scribe v2	18.5	14.8	8.9	28.6	3.4	21.2	26.5	17.2	12.0	24.7	27.2
AWS Transcribe	19.3	21.3	7.3	26.8	3.1	21.9	27.8	18.8	13.0	19.0	33.7
Google Gemini 3.5 Live	20.1	15.8	9.3	32.4	7.3	23.9	30.4	17.3	16.9	20.5	27.4
Speechmatics Enhanced	20.6	23.8	6.0	31.6	3.2	24.2	31.1	17.1	19.4	17.4	31.8
Meta Muse Voice Transcribe	22.2	23.9	8.4	38.5	10.0	20.0	20.0	17.4	27.8	25.6	30.5
Mistral Voxtral Mini	23.1	18.3	10.3	38.9	7.7	27.5	28.6	17.3	16.1	31.6	34.3
Reson8	23.7	18.6	8.9	30.8	11.9	31.8	26.6	18.7	20.9	38.0	31.3
xAI Grok Streaming	24.0	17.9	11.8	40.2	12.4	30.0	19.9	18.1	20.0	35.7	34.3
Deepgram Nova-3	26.1	24.3	11.0	30.4	4.5	34.4	43.7	16.8	27.6	29.1	39.4
Smallest Pulse	29.4	27.5	15.4	41.8	12.1	39.5	24.9	19.5	33.2	38.5	42.0
Azure	29.7	32.4	17.9	53.6	8.7	22.2	30.5	23.5	35.4	32.4	40.5
Deepgram Flux EN	30.1	29.0	13.2	41.0	11.5	40.7	22.8	21.0	46.1	32.3	43.9
Deepgram Flux Multi	30.4	32.1	14.2	41.7	9.3	36.2	22.3	22.3	46.0	37.1	43.1
Gladia Solaria	30.6	21.3	13.5	39.6	20.6	42.4	31.6	19.2	37.4	40.5	40.0
Speechmatics Standard	30.6	30.6	9.3	41.6	6.8	57.1	32.5	20.1	43.3	28.8	36.2
OpenAI GPT-4o Transcribe	38.1	27.8	17.9	51.7	42.8	50.3	33.2	30.5	47.7	37.3	41.8
OpenAI GPT-4o mini Transcribe	38.9	29.3	19.6	53.3	40.6	52.1	35.6	28.3	46.0	42.5	41.9
Cartesia Ink-Whisper	40.1	25.1	14.6	56.9	58.9	47.1	33.6	25.3	52.6	45.1	42.1

Best Worst Cells are coloured by rank within each column (solid blue is best, white is worst).

Background noise and utterance length

No background noise Natural low noise Natural heavy noise

Figure 2 Entity error rate (%) by background-noise condition, sorted by the no-noise value. We lead in all 3 conditions. normalized entity error rate (NEER) · runs 2026-08-27 → 2026-09-22 · one row per model, most recent run

Listen to the test data

One caller turn per noise environment, quietest first, each with an entity an agent acts on: a name, an account number, an e-mail login, a medication. Words that differ from the human reference are marked.

1 of 8

Voice agentsno noise4 s · AssemblyAI voice-agent corpus — scripted caller scenarios voiced by real speakers

0:00 / 0:00

Four seconds, no noise. A short request where the other systems substitute unrelated words.

Reference

My cable modem keeps rebooting every 30-40 minutes, can you help?

AssemblyAI

My cable modem keeps rebooting every 30 to 40 minutes. Can you help?

ElevenLabs Scribe v2 Realtime

My key won't let them keep changing every 30 to 40 minutes. Can you help?

Soniox Realtime

Mark Kilmer and keeps removing every 30 to 40 minutes. Can you help?

Short utterances Medium Long

Figure 3 Entity error rate (%) by utterance length. We lead in all 3 length bands. normalized entity error rate (NEER) · runs 2026-08-27 → 2026-09-22 · one row per model, most recent run

Speaker groups

Table 2 Entity error rate (%) by speaker group. Colour is rank within the column. We lead in all 12 groups. normalized entity error rate (NEER) · runs 2026-08-27 → 2026-09-22 · one row per model, most recent run

Entity error rate (%) by speaker group. Colour is rank within the column. We lead in all 12 groups.
System	US Northeast	US South	US Gen. Am.	US AAVE	US Asian-Am.	US other	UK	Canada	Native other	Spanish-acc.	Indian-acc.	Non-nat. other	meanrank
AssemblyAI Universal-3.6 Pro	10.1	11.5	11.4	12.4	12.0	13.9	14.0	10.5	15.2	16.1	19.1	18.9	1.0
AssemblyAI Universal-3.5 Pro	10.1	11.6	11.6	12.8	12.2	14.5	14.2	11.0	15.2	16.6	20.3	19.1	2.0
Zoom Scribe (Live)	13.8	15.8	15.0	15.3	15.8	20.4	18.7	13.5	19.1	20.5	23.9	24.1	3.4
ElevenLabs Scribe v2	14.6	15.8	15.0	16.6	16.1	19.4	19.5	13.7	19.8	22.9	23.8	25.4	4.2
Soniox Realtime	16.3	16.5	16.5	18.1	18.9	20.7	19.3	15.8	20.5	22.4	23.8	24.8	5.3
Google Gemini 3.5 Live	17.2	18.6	17.8	20.7	16.8	22.4	21.5	15.6	20.6	23.6	21.2	25.8	6.6
AWS Transcribe	17.7	18.4	19.2	20.7	19.8	22.4	20.1	18.0	21.6	23.8	24.4	27.4	7.7
xAI Grok Streaming	15.0	19.4	18.3	20.6	17.7	23.2	23.0	15.4	24.4	22.4	27.6	28.2	8.0
Mistral Voxtral Mini	18.0	19.0	18.3	20.4	19.6	22.4	20.6	15.5	23.6	28.7	28.0	31.1	8.8
Reson8	17.7	20.5	19.2	19.7	20.0	25.6	21.6	16.7	23.4	27.7	28.5	32.0	9.8
Meta Muse Voice Transcribe	19.5	20.3	20.6	21.5	21.1	22.7	22.7	19.2	24.1	28.0	26.6	29.6	10.5
Speechmatics Enhanced	21.4	21.0	21.2	22.0	22.1	24.8	22.6	19.8	23.8	27.0	26.6	29.6	10.8
Gladia Solaria	22.0	25.2	24.6	27.3	27.3	29.6	28.0	20.5	30.3	33.5	32.7	37.6	14.0
Deepgram Nova-3	24.4	26.4	26.2	27.0	27.2	28.4	28.6	24.3	30.1	31.0	32.9	35.1	14.3
Smallest Pulse	21.6	25.5	24.7	26.0	29.4	30.1	26.5	21.5	30.2	34.6	31.0	40.1	14.7
Deepgram Flux EN	24.4	26.5	26.0	27.5	27.1	30.4	32.2	25.7	32.3	33.7	34.9	37.6	15.8
Azure	26.6	29.0	28.4	29.0	28.9	32.1	31.4	26.0	31.9	36.4	35.8	37.8	17.5
Deepgram Flux Multi	25.3	27.2	27.1	29.2	27.7	32.6	33.0	26.3	33.3	36.1	36.3	39.8	17.8
Speechmatics Standard	26.7	28.9	28.4	31.6	32.0	33.4	31.5	28.2	33.5	37.4	35.7	39.7	18.8
OpenAI GPT-4o Transcribe	28.9	32.7	30.0	31.4	30.7	35.7	34.9	25.1	36.3	42.3	39.8	44.2	19.9
Cartesia Ink-Whisper	27.9	32.2	30.5	31.9	32.7	35.7	34.0	25.8	36.8	41.6	40.9	44.1	20.4
OpenAI GPT-4o mini Transcribe	29.4	33.0	30.5	32.3	32.4	37.0	35.3	27.5	37.5	42.9	41.6	46.9	21.8

Best Worst Cells are coloured by rank within each column (solid blue is best, white is worst).

Ablations on our own model

wordboost OFF wordboost ON

11.25%

Overall

4.36%

Names

6.14%

Places

28.16%

Addresses

2.69%

Phone

15.29%

20.01%

Dates

12.91%

Numbers

9.85%

Codes/IDs

7.16%

Medical

5.90%

Technical

5.05%

WER

Figure 4 Word boost (custom vocabulary) on versus off, same model and corpus. Change in entity error with word boost on: names −60.9%, technical terms −72.8%. normalized entity error rate (NEER) + WER, wordboost OFF vs ON · runs 2026-09-22 · one row per model, most recent run

agent context OFF agent context ON

15.35%

Overall

10.43%

Names

5.58%

Places

29.70%

Addresses

2.44%

Phone

16.30%

20.95%

Dates

15.61%

Numbers

10.45%

Codes/IDs

17.57%

Medical

24.40%

Technical

5.61%

WER

Figure 5 Agent context on versus off: the agent’s previous turn is passed to the recognizer. normalized entity error rate (NEER) + WER, same model with and without agent context · runs 2026-08-27 → 2026-08-28 · one row per model, most recent run

Word error rate

Table 3 WER by noise condition. normalized word error rate · runs 2026-08-27 → 2026-09-22 · one row per model, most recent run

WER by noise condition.
#	System model / snapshot	Mean 3 conditions	No noise	Low noise	Heavy noise
1	AssemblyAI Universal-3.6 Pro Realtime	5.19%	4.37%	5.08%	6.13%
2	AssemblyAI Universal-3.5 Pro Realtime	5.72%	4.61%	5.47%	7.08%
3	Soniox Realtime	6.83%	5.98%	6.84%	7.68%
4	Meta Muse Voice Transcribe	7.48%	6.92%	7.50%	8.02%
5	ElevenLabs Scribe v2 Realtime	7.78%	6.18%	7.53%	9.64%
6	Speechmatics Enhanced Realtime	8.14%	6.96%	7.96%	9.49%
7	Google Gemini 3.5 Transcribe Live	8.48%	7.44%	8.49%	9.52%
8	AWS Transcribe Streaming	8.58%	7.27%	8.27%	10.19%
9	Deepgram Nova-3	8.64%	7.53%	8.45%	9.93%
10	Zoom Scribe (Live)	9.36%	8.04%	9.29%	10.74%
11	Reson8 Reson8	9.41%	8.21%	9.38%	10.64%
12	Mistral Voxtral Mini Realtime	10.43%	9.25%	10.45%	11.59%
13	Smallest Pulse	10.74%	9.85%	10.94%	11.43%
14	Azure Realtime STT	10.86%	9.78%	10.65%	12.15%
15	Speechmatics Standard Realtime	12.05%	10.47%	11.77%	13.91%
16	Deepgram Flux EN	13.50%	12.29%	13.36%	14.85%
17	Deepgram Flux Multi	14.26%	12.85%	14.06%	15.87%
18	Cartesia Ink-Whisper	15.17%	13.71%	15.00%	16.81%
19	Gladia Solaria	16.25%	12.99%	16.04%	19.71%
20	OpenAI GPT-4o Transcribe	18.37%	17.05%	18.44%	19.63%
21	OpenAI GPT-4o mini Transcribe	19.55%	18.12%	19.77%	20.76%
22	xAI Grok Streaming	20.28%	18.45%	19.89%	22.50%

Best Others The best value in each column is filled blue.

Data

●Caller corpus 12,460 clips: All entity classes pooled into one number: how often an agent-critical value (a name, address, number or identifier) comes back wrong.
●Noise environments: Three levels of real noise, recorded rather than mixed in. None: room and handset noise only. Low: incidental noise from a home, office or parked car. Heavy: noise that competes with the voice.
●Utterance length: Short turns (confirmations, single values, one-word answers), medium and long. Short turns give the model almost no context.
●Speaker groups: Fourteen groups: US Northeast, South, General American, AAVE, Asian-American and other; UK; Canada; native speakers overall; Spanish- and Indian-accented and other non-native speakers.
●Entity classes: Names (including letter-by-letter spellings), places, street addresses, phone numbers, email addresses, dates and times, numbers and currency, codes and identifiers, medical and technical terms.

public, open source public, licensed private, ours

03Healthcare

Medical entity error is measured on primary-care consultations (PriMock57), licensed clinical dictation and respiratory-clinic recordings.

Overall

Table 4 Medical entity error rate. We lead on primary care. normalized medical entity error rate (MER) · runs 2026-08-18 → 2026-09-22 · one row per model, most recent run

Medical entity error rate. We lead on primary care.
#	System model / snapshot	Mean 3 sets	Primary care PriMock57	Dictation	Respiratory
1	Meta Muse Voice Transcribe	4.08%	7.18%	3.09%	1.97%
2	AssemblyAI Universal-3.6 Pro Realtime	4.24%	6.23%	4.20%	2.28%
3	Soniox Realtime	4.58%	7.18%	4.04%	2.51%
4	Speechmatics Enhanced Realtime	4.66%	8.13%	3.65%	2.21%
5	AssemblyAI Universal-3.5 Pro Realtime	4.81%	7.07%	5.08%	2.28%
6	ElevenLabs Scribe v2 Realtime	7.00%	10.89%	8.01%	2.09%
7	Google Gemini 3.5 Transcribe Live	8.57%	15.45%	7.69%	2.56%
8	AWS Transcribe Streaming	10.21%	11.09%	16.49%	3.06%
9	Zoom Scribe (Live)	10.50%	15.95%	8.09%	7.47%
10	Mistral Voxtral Mini Realtime	12.26%	12.67%	16.02%	8.09%
11	Deepgram Nova-3	13.22%	19.01%	17.37%	3.29%
12	Speechmatics Standard Realtime	14.24%	18.68%	18.72%	5.32%
13	Deepgram Flux EN	14.28%	20.49%	18.32%	4.02%
14	Reson8 Reson8	14.31%	16.37%	21.81%	4.76%
15	Azure Realtime STT	14.41%	17.21%	19.11%	6.92%
16	xAI Grok Streaming	15.69%	17.95%	19.35%	9.78%
17	Gladia Solaria	17.03%	18.83%	25.30%	6.97%
18	OpenAI GPT Realtime	17.21%	13.73%	5.39%	32.50%
19	Cartesia Ink-Whisper	21.09%	24.39%	31.09%	7.78%
20	Smallest Pulse	21.40%	25.42%	30.77%	8.01%
21	OpenAI GPT-4o Transcribe	26.56%	23.34%	14.91%	41.43%
22	OpenAI GPT-4o mini Transcribe	27.94%	25.45%	19.27%	39.11%

Best Others The best value in each column is filled blue.

Medication and disease

Table 5 Medication and disease terms scored separately. medication / disease entity error rate · runs 2026-08-18 → 2026-09-22 · one row per model, most recent run

Medication and disease terms scored separately.
#	System model / snapshot	Medication mean	Primary care	Dictation	Respiratory	Disease mean	Primary care	Dictation	Respiratory
1	Speechmatics Enhanced Realtime	6.62%	8.92%	5.04%	5.90%	3.38%	7.35%	0.93%	1.86%
2	Meta Muse Voice Transcribe	6.66%	10.40%	4.20%	5.38%	2.04%	3.99%	0.93%	1.19%
3	AssemblyAI Universal-3.6 Pro Realtime	7.11%	9.13%	5.52%	6.67%	2.11%	3.36%	1.64%	1.34%
4	Soniox Realtime	7.20%	10.83%	3.84%	6.92%	3.22%	3.57%	4.44%	1.64%
5	AssemblyAI Universal-3.5 Pro Realtime	7.37%	9.34%	6.60%	6.15%	2.78%	4.83%	2.10%	1.42%
6	ElevenLabs Scribe v2 Realtime	10.11%	13.57%	11.64%	5.13%	3.53%	8.17%	0.93%	1.49%
7	Google Gemini 3.5 Transcribe Live	12.23%	19.19%	10.80%	6.70%	5.16%	11.76%	1.64%	2.09%
8	AWS Transcribe Streaming	14.40%	15.07%	21.97%	6.15%	4.90%	7.14%	5.84%	1.72%
9	Zoom Scribe (Live)	15.21%	19.11%	11.40%	15.13%	6.31%	12.82%	1.64%	4.47%
10	Deepgram Nova-3	17.76%	25.05%	20.53%	7.69%	8.83%	13.03%	11.21%	2.24%
11	OpenAI GPT Realtime	17.76%	17.83%	6.72%	28.72%	14.02%	9.66%	2.80%	29.60%
12	Mistral Voxtral Mini Realtime	18.22%	18.47%	22.09%	14.10%	5.97%	6.93%	4.21%	6.79%
13	Azure Realtime STT	18.60%	23.57%	21.97%	10.26%	9.57%	10.92%	13.55%	4.25%
14	xAI Grok Streaming	19.50%	24.63%	26.05%	7.81%	7.74%	11.34%	6.31%	5.56%
15	Deepgram Flux EN	19.77%	29.30%	20.77%	9.23%	9.18%	11.76%	13.55%	2.24%
16	Speechmatics Standard Realtime	19.81%	26.98%	21.13%	11.31%	9.56%	10.53%	14.02%	4.13%
17	Reson8 Reson8	21.23%	23.78%	27.85%	12.05%	7.23%	9.00%	10.05%	2.61%
18	Gladia Solaria	23.23%	23.81%	32.41%	13.46%	9.69%	12.86%	11.45%	4.75%
19	Smallest Pulse	26.83%	31.55%	41.42%	7.53%	12.04%	18.30%	10.05%	7.76%
20	Cartesia Ink-Whisper	28.31%	33.55%	37.01%	14.36%	13.72%	15.34%	19.57%	6.26%
21	OpenAI GPT-4o Transcribe	28.38%	27.18%	16.93%	41.03%	22.68%	19.54%	10.98%	37.51%
22	OpenAI GPT-4o mini Transcribe	31.72%	32.91%	21.49%	40.77%	22.84%	18.07%	14.95%	35.50%

Best Others The best value in each column is filled blue.

Data

○Primary-care consultations (PriMock57) 114 consultations: Public corpus of mock primary-care consultations: clinicians and actor-patients work through realistic appointments, recorded as telemedicine calls.
◆Medical dictation 93 recordings: Licensed clinician dictation, notes and findings spoken for the record, dense with medication and disease terms.
●Respiratory clinic recordings 272 recordings: Real clinical conversation from respiratory clinic encounters, with the accents, interruptions and room noise of a working clinic.

public, open source public, licensed private, ours

04English

General English: entity accuracy and non-speech handling first, then word error rate on short-form, accented and long-form sets.

Table 6 Judge-scored entity error (%) on general English, by class. We lead on both overall scores. Per class, we lead on person names, organisations and URLs. A 0.00 cell means no data. LLM-judge-scored entity error rate (normalized and formatted) · runs 2026-08-18 → 2026-09-22 · one row per model, most recent run

Judge-scored entity error (%) on general English, by class. We lead on both overall scores. Per class, we lead on person names, organisations and URLs. A 0.00 cell means no data.
System	All classes	All, formatted	Persons	Products	Addresses	Orgs	Places	Phone	Email	URLs	Dates	Credit cards	Alphanum.
AssemblyAI Universal-3.5 Pro	18.1	27.4	19.9	19.2	21.5	16.4	20.5	9.3	16.7	15.1	16.5	19.6	9.0
AssemblyAI Universal-3.6 Pro	18.9	27.6	20.3	22.6	24.3	17.1	21.2	12.7	16.7	14.9	20.7	11.3	7.7
Google Gemini 3.5 Live	19.7	29.5	22.9	22.1	30.6	17.6	18.1	8.9	19.0	15.4	15.6	22.0	12.3
Speechmatics Enhanced	19.8	37.5	25.2	22.2	24.6	21.7	16.5	0.4	25.8	21.5	15.8	n/d	7.7
OpenAI GPT Realtime	20.2	38.6	24.8	27.5	21.4	20.0	17.7	2.1	19.9	21.9	19.0	3.0	13.4
ElevenLabs Scribe v2	20.4	29.2	24.8	22.6	29.2	22.6	18.9	n/d	17.2	14.9	19.4	n/d	8.7
Meta Muse Voice Transcribe	20.5	44.4	24.4	20.7	38.2	18.3	17.5	2.5	15.8	29.4	12.7	2.4	10.3
Mistral Voxtral Mini	20.8	31.7	27.8	24.0	26.4	19.3	17.4	3.8	13.1	30.0	16.4	n/d	7.5
Soniox Realtime	21.5	33.6	31.6	31.9	20.1	21.9	17.8	1.7	11.3	19.7	15.7	—	8.8
OpenAI GPT-4o Transcribe	24.4	37.8	26.2	25.9	33.0	23.3	26.2	18.1	25.3	19.7	21.9	21.4	10.5
Reson8	24.4	37.1	37.3	24.2	25.9	23.6	18.5	8.0	16.3	26.9	19.2	4.2	8.3
Gladia Solaria	24.9	41.2	32.5	26.6	29.2	24.1	22.1	4.2	21.3	31.0	20.2	7.7	10.3
Zoom Scribe (Live)	24.9	39.3	34.3	29.6	25.9	23.9	25.1	—	15.4	25.5	23.6	—	5.7
OpenAI GPT-4o mini Transcribe	25.0	38.1	28.7	27.1	33.1	24.0	27.2	8.9	24.0	20.1	20.2	8.3	12.6
Deepgram Nova-3	25.9	42.4	36.1	28.0	24.8	25.6	22.3	16.9	16.3	20.8	29.3	7.1	8.8
xAI Grok Streaming	26.1	50.2	34.0	28.8	27.6	24.8	26.1	2.5	19.9	28.3	24.2	4.8	8.5
Speechmatics Standard	26.6	44.4	33.9	25.5	29.1	26.2	20.4	6.3	30.8	39.8	16.5	n/d	26.5
Cartesia Ink-Whisper	27.8	43.0	37.6	28.2	32.9	27.3	26.1	12.7	18.9	24.2	20.9	13.7	10.7
AWS Transcribe	28.1	43.8	46.2	29.6	20.4	27.6	23.4	0.4	20.8	28.3	17.3	8.3	7.5
Deepgram Flux EN	31.4	62.5	49.1	33.0	32.7	30.2	25.0	3.4	15.8	31.4	23.5	n/d	10.0
Azure	33.1	97.5	45.0	35.9	50.2	34.4	27.8	2.1	14.5	24.7	31.0	0.6	10.0
Smallest Pulse	33.2	58.0	53.0	33.0	33.0	31.7	28.7	3.8	17.6	25.3	21.1	5.4	10.0

Best Worst Cells are coloured by rank within each column (solid blue is best, white is worst).

Figure 6 Non-speech response rate: how often a system produces text for a clip with no speech. Lower is better. The axis is clipped at 5%; the notched bar exceeds it. non-speech response rate (share of pure non-speech clips that produce any text) · runs 2026-08-18 → 2026-09-22 · one row per model, most recent run

Word error rate — short-form

Table 7 Short-form English WER. We lead on the mean and Common Voice. Systems without a Common Voice run have no mean. normalized word error rate · runs 2026-08-18 → 2026-09-22 · one row per model, most recent run

Short-form English WER. We lead on the mean and Common Voice. Systems without a Common Voice run have no mean.
#	System model / snapshot	Mean 3 sets	Common Voice	LibriSpeech clean	LibriSpeech other
1	AssemblyAI Universal-3.6 Pro Realtime	3.44%	5.18%	1.88%	3.25%
2	Reson8 Reson8	3.70%	6.98%	1.43%	2.68%
3	Zoom Scribe (Live)	3.75%	6.32%	1.72%	3.21%
4	AssemblyAI Universal-3.5 Pro Realtime	3.83%	6.44%	1.83%	3.22%
5	AWS Transcribe Streaming	3.93%	6.12%	1.90%	3.77%
6	Speechmatics Enhanced Realtime	4.69%	6.78%	2.31%	4.99%
7	Google Gemini 3.5 Transcribe Live	5.04%	8.22%	2.07%	4.83%
8	ElevenLabs Scribe v2 Realtime	5.13%	9.01%	1.96%	4.43%
9	Smallest Pulse	5.41%	9.34%	2.19%	4.69%
10	Azure Realtime STT	5.54%	8.68%	2.44%	5.49%
11	xAI Grok Streaming	5.81%	11.04%	2.04%	4.36%
12	Meta Muse Voice Transcribe	6.20%	11.24%	2.15%	5.22%
13	Speechmatics Standard Realtime	6.41%	9.43%	3.18%	6.61%
14	Deepgram Flux EN	6.53%	7.68%	3.56%	8.35%
15	Mistral Voxtral Mini Realtime	6.61%	12.23%	2.10%	5.49%
16	Gladia Solaria	7.01%	12.88%	2.73%	5.42%
17	Deepgram Nova-3	7.46%	12.38%	3.28%	6.72%
18	Soniox Realtime	7.60%	12.38%	3.27%	7.15%
19	Cartesia Ink-Whisper	7.66%	13.31%	3.19%	6.48%
20	OpenAI GPT Realtime	7.95%	13.83%	2.59%	7.44%
21	OpenAI GPT-4o Transcribe	8.11%	14.99%	2.20%	7.15%
22	OpenAI GPT-4o mini Transcribe	8.75%	15.76%	2.53%	7.97%

Best Others The best value in each column is filled blue.

Listen to the test data

Two read sentences from the short-form sets above, human reference beside each transcript. Words that differ from it are marked.

1 of 2

Short-form English17 s · Common Voice (CC0)

0:00 / 0:00

A read sentence. The other system appends a sentence and a name that are not in the audio.

Reference

You can do a lot just using your voice, but there are still a few times you'll find yourself reaching for a mouse.

AssemblyAI

You can do a lot just using your voice, but there are still a few times you'll find yourself reaching for a mouse.

Speechmatics Enhanced

I know you can do a lot just using your voice, but there are still a few times you'll find yourself reaching for a mouse. And I'm Larry Hanson. We're here in this beautiful hall.

Word error rate — accented English

Table 8 Accented English WER. We lead on British-accented English and Indian-accented English. normalized word error rate · runs 2026-08-18 → 2026-09-22 · one row per model, most recent run

Accented English WER. We lead on British-accented English and Indian-accented English.
#	System model / snapshot	Mean 4 sets	British	French-acc.	Spanish-acc.	Indian-acc.
1	Azure Realtime STT	6.09%	6.25%	4.44%	5.52%	8.15%
2	AssemblyAI Universal-3.6 Pro Realtime	7.29%	5.53%	8.54%	9.62%	5.47%
3	AssemblyAI Universal-3.5 Pro Realtime	7.46%	5.72%	8.69%	9.53%	5.89%
4	Zoom Scribe (Live)	7.94%	6.14%	9.57%	10.03%	6.04%
5	Reson8 Reson8	9.24%	5.76%	11.89%	12.60%	6.73%
6	Google Gemini 3.5 Transcribe Live	9.39%	6.30%	11.86%	12.82%	6.59%
7	Meta Muse Voice Transcribe	9.64%	7.16%	11.26%	12.22%	7.93%
8	xAI Grok Streaming	9.85%	7.33%	12.21%	12.88%	6.99%
9	ElevenLabs Scribe v2 Realtime	10.10%	6.92%	12.12%	13.29%	8.07%
10	OpenAI GPT-4o Transcribe	10.30%	7.27%	11.83%	13.89%	8.22%
11	Speechmatics Enhanced Realtime	10.44%	6.70%	13.26%	14.80%	6.98%
12	Gladia Solaria	11.07%	8.67%	12.60%	13.85%	9.15%
13	AWS Transcribe Streaming	11.10%	7.02%	15.49%	15.47%	6.42%
14	Cartesia Ink-Whisper	11.23%	8.16%	12.92%	14.57%	9.27%
15	OpenAI GPT-4o mini Transcribe	11.36%	7.88%	13.31%	15.35%	8.88%
16	Soniox Realtime	11.69%	8.99%	12.87%	14.32%	10.58%
17	OpenAI GPT Realtime	11.78%	8.31%	14.83%	15.78%	8.20%
18	Smallest Pulse	11.85%	7.70%	15.15%	16.53%	8.03%
19	Deepgram Nova-3	11.95%	9.14%	13.92%	15.19%	9.57%
20	Deepgram Flux EN	12.55%	9.61%	14.83%	16.12%	9.64%
21	Speechmatics Standard Realtime	12.58%	8.43%	16.29%	17.46%	8.14%
22	Mistral Voxtral Mini Realtime	12.63%	7.39%	14.19%	15.87%	13.06%

Best Others The best value in each column is filled blue.

Word error rate — long-form

Table 9 Long-form English WER. TED-LIUM references omit sponsor read-outs present in the audio. normalized word error rate · runs 2026-08-18 → 2026-09-22 · one row per model, most recent run

Long-form English WER. TED-LIUM references omit sponsor read-outs present in the audio.
#	System model / snapshot	Mean 3 sets	TED-LIUM	Rev16	CALLHOME per channel
1	Meta Muse Voice Transcribe	7.37%	6.01%	6.57%	8.36%
2	Reson8 Reson8	7.74%	6.01%	6.87%	9.17%
3	Speechmatics Enhanced Realtime	8.08%	6.29%	7.34%	9.59%
4	Soniox Realtime	8.19%	7.13%	8.17%	9.46%
5	AWS Transcribe Streaming	8.71%	6.45%	7.81%	9.82%
6	AssemblyAI Universal-3.6 Pro Realtime	8.82%	8.14%	7.97%	11.07%
7	Speechmatics Standard Realtime	8.90%	6.83%	8.06%	10.77%
8	Azure Realtime STT	9.20%	7.16%	8.17%	10.87%
9	Zoom Scribe (Live)	9.34%	7.41%	7.17%	13.26%
10	AssemblyAI Universal-3.5 Pro Realtime	9.35%	8.58%	8.24%	11.69%
11	Google Gemini 3.5 Transcribe Live	9.42%	6.87%	8.04%	10.99%
12	xAI Grok Streaming	9.43%	6.30%	8.00%	12.82%
13	Deepgram Flux EN	9.59%	7.01%	8.08%	12.12%
14	Deepgram Nova-3	9.75%	7.43%	9.40%	11.19%
15	Cartesia Ink-Whisper	10.60%	7.13%	10.42%	15.62%
16	Gladia Solaria	10.72%	7.75%	9.04%	18.25%
17	Smallest Pulse	10.95%	5.23%	7.62%	14.13%
18	OpenAI GPT Realtime	15.87%	5.70%	6.02%	20.90%
19	ElevenLabs Scribe v2 Realtime	17.12%	5.96%	8.38%	45.51%
20	Mistral Voxtral Mini Realtime	18.93%	6.96%	8.33%	52.91%
21	OpenAI GPT-4o mini Transcribe	19.10%	7.28%	21.26%	18.56%
22	OpenAI GPT-4o Transcribe	27.45%	9.88%	28.51%	30.33%

Best Others The best value in each column is filled blue.

Data

●Judge-scored entities: The short-form English audio below. An LLM judge grades each entity (person, product, organisation, place, address, phone, email, date, ID, URL) on whether the transcript names the same real-world thing, with and without formatting.
●Non-speech 2,950 clips: Hold music, line noise and silence, with no words at all. The score is how often a model emits text anyway.
○Common Voice 16,029 clips: Crowdsourced read sentences from Mozilla’s public Common Voice, recorded by volunteers on their own phones and laptops, so accents, microphones and rooms vary widely.
○LibriSpeech test-clean 2,620 utterances: Read English audiobook speech from the public LibriSpeech corpus, recorded close-mic by LibriVox volunteers. The "clean" split is the easier half: clear speakers, little noise.
○LibriSpeech test-other 2,939 utterances: The harder half of the same LibriSpeech set: speakers and recordings that automatic transcription finds more difficult, still read prose.
◆Accented English 9,315 files: Four licensed sets of prompted British-, French-, Spanish- and Indian-accented English: accuracy outside the General American accent most models are tuned on.
○TED-LIUM 11 full-length talks: Full TED talks from the public TED-LIUM corpus: one prepared speaker on stage, recorded end to end, many minutes of continuous speech per file.
○Rev16 9 full-length recordings: Podcast-style recordings from the public Rev16 set: spontaneous multi-speaker studio conversation, episode length.
◆CALLHOME 71 scored channels: Public CALLHOME telephone calls between friends and family: unscripted, fast, overlapping conversation over a real phone line. Each speaker's channel is scored on its own.

public, open source public, licensed private, ours

05Multilingual

Eighteen languages on FLEURS and Common Voice, drawn from the thirty-two Universal-3.6 Pro Realtime supports. A system is scored on every language it supports, and a cross marks one its vendor does not document for realtime use — so the means below cover different numbers of languages and the Languages column says how many.

FLEURS — Systems covering all 18 languages

Table 10 WER on FLEURS (CER for languages marked *). Each mean covers only the languages that system supports; the Languages column gives the count. normalized word error rate (character error rate for Japanese, Chinese, Korean) · runs 2026-08-25 → 2026-09-23 · one row per model, most recent run

WER on FLEURS (CER for languages marked *). Each mean covers only the languages that system supports; the Languages column gives the count.
System	Mean over supported langs	Languages of 18	Spanish	French	German	Italian	Portuguese	Dutch	Danish	Swedish	Finnish	Russian	Turkish	Arabic	Hebrew	Hindi	Vietnamese	Japanese*	Chinese*	Korean*
Google Gemini 3.5 Live	6.4	18	3.8	5.4	4.7	3.2	4.7	5.0	9.1	7.3	7.5	5.3	5.3	6.7	18.9	5.0	5.2	5.0	9.4	3.8
Soniox Realtime	7.9	18	5.0	7.8	6.9	4.9	6.2	7.7	11.4	10.3	8.0	7.3	7.3	7.2	19.2	6.3	7.0	5.8	10.3	4.4
AssemblyAI Universal-3.6 Pro	8.4	18	4.1	5.0	5.0	4.0	5.5	5.9	12.7	6.6	10.8	7.5	8.6	12.1	18.2	7.5	13.9	6.3	14.0	4.3
AWS Transcribe	8.9	18	5.3	7.6	8.0	4.6	8.2	7.1	11.5	10.0	9.8	9.4	8.6	8.5	22.6	6.3	9.3	7.0	12.3	4.9
ElevenLabs Scribe v2	9.3	18	4.6	6.9	5.8	4.3	6.4	7.0	12.5	10.9	9.5	6.6	7.7	10.8	27.0	15.8	8.8	8.6	10.1	4.6
Meta Muse Voice Transcribe	10.0	18	6.5	8.0	6.4	6.0	8.7	8.6	13.6	11.0	10.7	10.1	10.5	8.9	27.0	8.1	6.5	9.8	13.8	6.1
Speechmatics Enhanced	11.1	18	4.7	6.5	6.8	5.4	7.3	8.5	11.8	9.9	8.0	10.0	9.1	6.5	18.5	7.0	8.6	31.3	33.8	5.4
Speechmatics Standard	11.6	18	5.4	8.1	8.2	5.7	7.3	9.5	11.7	10.2	8.7	10.5	8.7	8.4	19.3	7.0	9.3	31.4	34.7	5.2
OpenAI GPT Realtime	12.3	18	5.7	11.8	11.2	5.9	8.8	10.1	14.7	12.4	12.4	10.4	14.3	10.5	34.5	10.4	11.4	13.4	16.3	7.1
Azure	12.7	18	7.7	11.4	9.1	8.3	10.9	12.9	19.8	16.5	15.6	13.1	15.2	12.5	27.6	8.6	9.4	8.6	11.5	9.4
OpenAI GPT-4o Transcribe	12.8	18	4.3	10.6	13.1	5.5	8.5	13.0	17.7	10.1	9.2	8.4	14.8	9.4	34.7	12.4	18.7	16.1	17.4	6.7
OpenAI GPT-4o mini Transcribe	15.6	18	5.5	13.2	14.9	6.9	10.9	16.8	20.6	12.7	12.4	9.9	17.5	11.7	45.0	16.8	23.1	15.4	19.3	7.6
Gladia Solaria	16.2	18	6.7	12.7	10.9	7.5	8.7	11.6	22.6	15.3	15.9	10.3	12.2	15.1	45.4	23.9	21.5	23.6	21.1	6.6
Cartesia Ink-Whisper	18.7	18	9.0	16.9	12.7	9.7	10.7	17.4	25.4	16.1	17.0	15.7	16.2	18.1	42.0	27.5	23.6	20.4	26.3	10.9

Best Worst Cells are coloured by rank within each column (solid blue is best, white is worst).

FLEURS — Systems covering fewer than 18

Table 11 The same measurement on the systems that do not cover all eighteen. Each mean is over that system’s own languages, so these are not comparable with the table above or with each other — the Languages column gives the count, and a cross marks a language the vendor does not document for realtime use. normalized word error rate (character error rate for Japanese, Chinese, Korean) · runs 2026-08-25 → 2026-09-23 · one row per model, most recent run

The same measurement on the systems that do not cover all eighteen. Each mean is over that system’s own languages, so these are not comparable with the table above or with each other — the Languages column gives the count, and a cross marks a language the vendor does not document for realtime use.
System	Mean over supported langs	Languages of 18	Spanish	French	German	Italian	Portuguese	Dutch	Danish	Swedish	Finnish	Russian	Turkish	Arabic	Hebrew	Hindi	Vietnamese	Japanese*	Chinese*	Korean*
Reson8	5.8	7	4.0	5.5	5.3	4.0	5.6	6.3	✕	9.6	✕	✕	✕	✕	✕	✕	✕	✕	✕	✕
Zoom Scribe (Live)	6.2	7	5.2	5.9	6.5	5.2	6.7	✕	✕	✕	✕	✕	✕	7.9	✕	✕	✕	6.2	—	✕
Mistral Voxtral Mini	9.5	12	4.8	9.6	7.8	5.5	6.9	9.5	—	—	—	8.2	—	12.7	—	16.8	—	9.6	16.5	5.8
AssemblyAI Universal-3.5 Pro	9.9	16	4.4	5.2	5.3	4.3	5.8	6.5	13.1	7.5	12.5	—	9.8	14.0	21.9	12.4	15.9	6.5	13.4	—
Smallest Pulse	11.9	11	8.9	12.7	11.6	7.0	11.3	13.1	—	✕	✕	15.8	—	—	—	10.7	—	12.7	20.1	7.4
Deepgram Flux	12.7	9	7.8	13.9	10.2	8.8	12.2	15.8	✕	✕	✕	11.5	✕	✕	✕	20.1	✕	13.6	✕	✕
Deepgram Nova-3	14.2	9	8.9	14.0	13.3	10.0	13.3	14.6	—	—	—	12.5	—	—	—	22.3	—	18.8	—	—
xAI Grok Streaming	14.8	14	6.1	26.0	10.9	6.2	8.4	12.5	28.3	23.2	✕	12.0	15.4	15.4	✕	—	12.9	21.7	✕	7.8

Best Worst Cells are coloured by rank within each column (solid blue is best, white is worst).

Common Voice — Systems covering all 18 languages

Table 12 WER on Common Voice (CER for languages marked *). Each mean covers only the languages that system supports; the Languages column gives the count. normalized word error rate (character error rate for Japanese, Chinese, Korean) · runs 2026-08-25 → 2026-09-23 · one row per model, most recent run

WER on Common Voice (CER for languages marked *). Each mean covers only the languages that system supports; the Languages column gives the count.
System	Mean over supported langs	Languages of 18	Spanish	French	German	Italian	Portuguese	Dutch	Danish	Swedish	Finnish	Russian	Turkish	Arabic	Hebrew	Hindi	Vietnamese	Japanese*	Chinese*	Korean*
Speechmatics Enhanced	7.3	18	2.5	7.2	3.7	4.7	5.2	2.4	7.3	4.0	3.4	7.1	7.3	6.1	9.4	3.9	2.8	23.2	25.1	7.2
AWS Transcribe	7.5	18	4.1	8.8	5.8	4.2	4.9	2.7	7.3	5.9	5.7	4.8	10.3	9.6	15.0	5.1	8.2	15.9	8.5	7.5
AssemblyAI Universal-3.6 Pro	7.6	18	3.2	6.9	3.6	3.3	4.9	2.6	9.4	4.8	9.3	8.1	11.0	9.4	12.3	5.3	10.5	16.2	9.2	6.8
Speechmatics Standard	9.0	18	4.2	10.2	6.2	5.3	6.4	3.0	8.1	5.7	5.9	7.7	6.5	11.1	11.8	4.0	6.2	25.4	25.8	8.3
Google Gemini 3.5 Live	9.2	18	4.0	9.3	5.2	4.2	5.8	3.3	11.5	9.7	11.5	6.5	10.1	9.6	17.8	4.7	9.3	21.7	13.3	7.5
Soniox Realtime	9.5	18	6.0	12.2	7.6	6.3	7.4	4.6	9.5	8.6	7.7	9.0	10.8	10.2	12.5	6.6	10.1	19.2	14.1	8.7
Azure	10.0	18	6.8	11.0	6.1	6.2	7.2	4.6	11.9	10.0	11.1	7.0	12.3	11.5	20.0	4.9	8.8	19.8	9.9	10.4
Meta Muse Voice Transcribe	10.2	18	4.9	10.4	6.6	5.5	6.4	5.7	14.6	11.1	13.6	7.2	9.6	9.3	18.4	5.8	9.0	20.3	16.4	9.6
OpenAI GPT Realtime	12.3	18	6.9	12.6	9.3	7.8	9.4	6.8	14.5	12.0	11.8	8.2	11.1	12.7	28.9	8.4	13.4	20.2	16.4	10.5
ElevenLabs Scribe v2	14.6	18	5.8	10.2	7.1	6.1	9.7	5.5	14.5	12.9	14.6	10.2	15.0	20.6	43.4	21.6	21.5	20.7	12.1	11.0
OpenAI GPT-4o Transcribe	15.0	18	8.9	15.5	10.3	9.2	13.2	7.8	18.0	12.6	11.2	9.3	15.1	17.2	34.9	10.0	20.8	22.0	21.4	12.1
Gladia Solaria	16.1	18	7.8	14.4	10.5	9.8	9.5	5.7	17.5	12.6	14.0	9.5	16.2	19.8	36.1	22.0	20.6	26.2	29.0	9.6
OpenAI GPT-4o mini Transcribe	17.7	18	9.8	16.8	11.7	10.6	14.3	11.9	21.9	17.1	13.3	11.9	14.3	22.5	42.9	21.5	23.9	22.3	19.3	12.2
Cartesia Ink-Whisper	18.9	18	10.4	17.7	12.4	11.1	11.5	9.7	21.1	16.3	17.7	12.3	18.0	22.0	39.1	27.9	22.2	28.3	29.2	13.8

Best Worst Cells are coloured by rank within each column (solid blue is best, white is worst).

Common Voice — Systems covering fewer than 18

Table 13 The same measurement on the systems that do not cover all eighteen. Each mean is over that system’s own languages, so these are not comparable with the table above or with each other — the Languages column gives the count, and a cross marks a language the vendor does not document for realtime use. normalized word error rate (character error rate for Japanese, Chinese, Korean) · runs 2026-08-25 → 2026-09-23 · one row per model, most recent run

The same measurement on the systems that do not cover all eighteen. Each mean is over that system’s own languages, so these are not comparable with the table above or with each other — the Languages column gives the count, and a cross marks a language the vendor does not document for realtime use.
System	Mean over supported langs	Languages of 18	Spanish	French	German	Italian	Portuguese	Dutch	Danish	Swedish	Finnish	Russian	Turkish	Arabic	Hebrew	Hindi	Vietnamese	Japanese*	Chinese*	Korean*
Reson8	4.9	7	3.5	7.2	4.1	3.1	4.3	2.9	✕	9.2	✕	✕	✕	✕	✕	✕	✕	✕	✕	✕
Zoom Scribe (Live)	8.0	7	5.7	8.6	5.8	4.1	6.6	✕	✕	✕	✕	✕	✕	8.6	✕	✕	✕	16.4	—	✕
AssemblyAI Universal-3.5 Pro	10.5	16	4.3	8.3	5.0	4.4	8.0	3.6	11.0	6.8	15.2	—	15.5	14.1	14.9	10.6	13.8	19.6	13.1	—
Smallest Pulse	12.4	11	8.6	12.3	11.3	6.6	21.6	9.2	—	✕	✕	10.8	—	—	—	9.7	—	17.5	18.0	10.7
Deepgram Flux	13.6	9	7.8	13.7	9.9	10.8	10.5	9.1	✕	✕	✕	13.1	✕	✕	✕	22.7	✕	24.4	✕	✕
Mistral Voxtral Mini	13.7	12	5.7	11.5	8.8	7.4	9.9	8.5	—	—	—	9.8	—	17.4	—	18.1	—	22.8	30.5	14.3
Deepgram Nova-3	15.2	9	8.4	13.8	12.8	10.8	12.7	9.3	—	—	—	14.4	—	—	—	25.2	—	29.7	—	—
xAI Grok Streaming	17.3	14	8.9	13.7	11.7	9.1	11.1	9.5	30.8	28.8	✕	13.7	23.7	24.2	✕	—	17.8	25.0	✕	13.5

Best Worst Cells are coloured by rank within each column (solid blue is best, white is worst).

Data

○FLEURS: Public corpus of parallel sentences read by native speakers in quiet settings, one test set per language. Shown: Spanish, French, German, Italian, Portuguese, Dutch, Hindi, Japanese and Chinese.
○Common Voice: Mozilla’s public Common Voice, per language: volunteers reading sentences on their own devices, so accents and microphones vary. Japanese and Chinese are scored by character error rate.

public, open source public, licensed private, ours

06Code-switching

Speech that switches between English and a second language within one utterance: five language pairs plus a Spanish–English conversational corpus.

Table 14 Code-switching WER. We lead on the five-pair mean. We lead in all 5 pairs. normalized word error rate · runs 2026-08-18 → 2026-09-23 · one row per model, most recent run · pair value = mean of 4 sub-sets

Code-switching WER. We lead on the five-pair mean. We lead in all 5 pairs.
#	System model / snapshot	Mean 5 pairs	EN↔ES	EN↔DE	EN↔FR	EN↔IT	EN↔PT	Miami 10h
1	AssemblyAI Universal-3.6 Pro Realtime	7.20%	9.59%	6.13%	7.23%	6.03%	7.00%	24.28%
2	AssemblyAI Universal-3.5 Pro Realtime	8.65%	10.95%	7.84%	8.54%	7.21%	8.71%	27.31%
3	AWS Transcribe Streaming	9.13%	11.32%	8.12%	9.33%	7.05%	9.81%	27.96%
4	Google Gemini 3.5 Transcribe Live	10.16%	14.40%	8.29%	10.05%	8.25%	9.79%	40.69%
5	Reson8 Reson8	10.67%	12.58%	9.73%	12.60%	8.77%	9.67%	24.48%
6	Soniox Realtime	11.26%	12.12%	10.81%	12.15%	10.04%	11.18%	21.93%
7	ElevenLabs Scribe v2 Realtime	13.01%	14.27%	12.29%	13.16%	11.53%	13.79%	27.09%
8	Meta Muse Voice Transcribe	13.34%	13.34%	12.31%	14.64%	11.56%	14.85%	23.10%
9	Smallest Pulse	14.56%	18.52%	13.14%	13.61%	10.79%	16.73%	47.17%
10	Azure Realtime STT	14.60%	17.25%	13.33%	14.86%	12.52%	15.03%	34.17%
11	Gladia Solaria	17.09%	18.39%	18.10%	18.73%	14.28%	15.93%	34.49%
12	Deepgram Nova-3 Multi	17.13%	16.66%	18.01%	18.52%	15.17%	17.29%	29.91%
13	Mistral Voxtral Mini Realtime	18.46%	18.37%	19.11%	20.79%	16.61%	17.43%	34.35%
14	xAI Grok Streaming	20.61%	21.73%	20.06%	21.69%	18.79%	20.79%	34.20%
15	Zoom Scribe (Live)	32.48%	26.66%	35.09%	32.56%	32.09%	36.02%	37.87%
16	OpenAI GPT-4o mini Transcribe	33.27%	35.28%	33.06%	34.22%	31.73%	32.07%	45.01%
17	OpenAI GPT-4o Transcribe	35.46%	39.66%	34.70%	35.78%	33.17%	34.00%	61.27%
18	OpenAI GPT Realtime	37.35%	39.34%	35.46%	37.87%	36.30%	37.77%	53.31%
19	Speechmatics Enhanced Realtime	38.23%	33.75%	33.78%	35.98%	45.39%	42.23%	36.04%
20	Speechmatics Standard Realtime	42.86%	37.60%	44.12%	42.99%	45.11%	44.49%	40.95%
21	Cartesia Ink-Whisper EN	50.52%	46.90%	51.50%	53.10%	53.17%	47.95%	43.72%
22	Deepgram Flux EN	52.86%	52.36%	52.33%	56.46%	53.39%	49.76%	45.55%
23	Deepgram Nova-3 EN	54.71%	54.39%	54.61%	58.15%	56.24%	50.15%	43.85%

Best Others The best value in each column is filled blue.

Figure 7 Five-pair mean WER. normalized word error rate · runs 2026-08-18 → 2026-09-23 · one row per model, most recent run

Data

○English ↔ Spanish, German, French, Italian, Portuguese 5 pairs · 4 sets each: Audio that alternates between English and the second language mid-stream, built from public read-speech corpora in short, long and split-long clips so the switch falls at different points.
◆Miami, 10-hour selection 21 recordings: A ten-hour selection of the Miami Spanish-English conversation, with references produced independently by a third-party transcription service.

public, open source public, licensed private, ours

07Diarization

Speaker attribution on CALLHOME, AMI, DiPCo and NOTSOFAR, for systems that offer streaming diarization. Diarization error rate is the share of audio time given to the wrong speaker, missed, or wrongly marked as speech.

Table 15 Diarization error rate and cpWER. Reson8 caps speaker count at 4, its product maximum. That fits CALLHOME, AMI and DiPCo, whose references never exceed four speakers, and costs it on NOTSOFAR, which runs to seven, so its per-corpus figures are not directly comparable with systems that estimate speaker count freely. diarization error rate (DER) and cpWER · runs 2026-08-20 → 2026-09-24 · one row per model, most recent run

Diarization error rate and cpWER. Reson8 caps speaker count at 4, its product maximum. That fits CALLHOME, AMI and DiPCo, whose references never exceed four speakers, and costs it on NOTSOFAR, which runs to seven, so its per-corpus figures are not directly comparable with systems that estimate speaker count freely.
#	System model / snapshot	DER mean 4 sets	CALLHOME	AMI	DiPCo	NOTSOFAR	cpWER mean	CALLHOME	AMI	DiPCo	NOTSOFAR
1	Meta Muse Voice Transcribe	14.51%	16.44%	19.45%	14.81%	7.34%	24.23%	12.54%	22.66%	26.65%	35.07%
2	AssemblyAI Universal-3.5 Pro Realtime	15.96%	13.87%	18.31%	21.29%	10.37%	30.59%	19.76%	28.00%	34.00%	40.61%
3	AssemblyAI Universal-3.6 Pro Realtime	16.68%	13.74%	18.79%	22.10%	12.09%	30.92%	19.21%	28.16%	34.20%	42.11%
4	Reson8 Reson8	16.72%	15.99%	15.93%	15.98%	18.98%	34.32%	22.34%	28.21%	34.61%	52.11%
5	Azure Realtime STT	20.43%	20.21%	18.00%	24.30%	19.19%	40.96%	34.68%	33.63%	41.45%	54.06%
6	Speechmatics Enhanced Realtime	22.16%	14.58%	19.96%	28.37%	25.74%	37.14%	24.69%	23.01%	42.59%	58.25%
7	xAI Grok Streaming	25.68%	15.59%	31.90%	35.28%	19.96%	41.81%	25.77%	37.95%	44.72%	58.82%
8	Speechmatics Standard Realtime	25.71%	25.43%	21.27%	24.68%	31.47%	48.86%	47.84%	33.66%	43.43%	70.49%
9	AWS Transcribe Streaming	27.48%	27.89%	26.16%	32.09%	23.79%	46.07%	41.66%	38.72%	47.91%	55.98%
10	Soniox Realtime	31.77%	19.83%	46.27%	27.23%	33.76%	42.78%	17.20%	55.47%	39.56%	58.88%
11	Deepgram Nova-3	34.01%	30.46%	33.86%	45.35%	26.37%	56.09%	52.17%	43.99%	69.88%	58.30%
12	Smallest Pulse	41.35%	27.16%	59.41%	50.98%	27.87%	53.34%	31.65%	65.89%	62.56%	53.26%

Best Others The best value in each column is filled blue.

Data

◆CALLHOME 40 calls: Public CALLHOME telephone calls between friends and family: unscripted, fast, heavily overlapping conversation over a real phone line.
○AMI 24 meetings: Public AMI Meeting Corpus: small groups of colleagues in real design and project meetings, in instrumented rooms.
○NOTSOFAR 129 meetings: Public NOTSOFAR corpus: real multi-party meetings on a single distant microphone in ordinary meeting rooms.
○DiPCo 10 sessions: Public Dinner Party Corpus: groups talking over dinner on far-field microphones, constant overlap and background clatter.

public, open source public, licensed private, ours

08Independent evaluations

Third-party results on their own corpora and pipelines, reproduced for comparison.

Artificial Analysis — Speech to Text (Streaming)

AA-WER v2.2 (May 2026): 50% AA-AgentTalk, 25% VoxPopuli, 25% Earnings22. End of speech is detected with SileroVAD; latency runs from there to the final transcript. Price is per 1,000 minutes of audio, normalized across billing models. The leaderboard publishes no “last updated” date. The two AssemblyAI configurations shown are Universal-3.5 Pro Realtime. Universal-3.6 Pro has not been scored here yet; this section will be updated once the source publishes it.

Table 16 30 of the 31 systems on the Artificial Analysis streaming leaderboard, at published ranks; our earlier-generation model is omitted. source = https://artificialanalysis.ai/speech-to-text/streaming · read 2026-09-09 · price shown for every system the source publishes one for

30 of the 31 systems on the Artificial Analysis streaming leaderboard, at published ranks; our earlier-generation model is omitted.
#	System	AA-WER index %	t final s	Price $/1k min
1	Muse Voice Transcribe (Meta)	3.06%	0.163 s	3.0
2	Cartesia Ink-2 (semantic)	3.36%	0.431 s	4.0
3	ElevenLabs Scribe v2 Realtime	3.59%	0.141 s	6.5
4	Qwen3 ASR Flash Realtime	3.73%	0.476 s	5.4
5	GPT Live Transcribe	3.92%	0.812 s	17.0
6	Grok STT Streaming	3.93%	0.373 s	3.3
7	Gemini 3.5 Transcribe Live	4.00%	0.395 s	9.0
8	AssemblyAI U3.5 Realtime Pro — min latency	4.02%	0.191 s	7.5
9	Cartesia Ink-2 (external endpoint)	4.02%	0.067 s	4.0
10	AssemblyAI U3.5 Realtime Pro — max accuracy	4.03%	0.202 s	7.5
12	Inworld STT 1	4.18%	0.071 s	1.4
13	Soniox v4	4.49%	0.062 s	2.0
14	Soniox v5	4.50%	0.054 s	2.0
15	Google Chirp 3 Streaming	4.80%	1.276 s	16.0
16	OpenAI GPT Realtime (Whisper)	4.89%	0.688 s	17.0
17	Smallest Pulse	4.98%	0.132 s	4.0
18	Voxtral Mini Realtime	5.24%	0.682 s	6.0
19	Azure Speech	5.25%	0.625 s	16.7
20	Nemotron 3 ASR (1120 ms)	5.36%	0.418 s	—
21	Amazon Transcribe	5.79%	0.620 s	24.0
22	Nemotron 3 ASR (560 ms)	5.96%	0.252 s	—
23	Deepgram Nova-3 Realtime	6.59%	0.066 s	4.8
24	Gradium	6.61%	0.435 s	12.0
25	Nemotron 3 ASR (80 ms, Together)	7.03%	0.070 s	—
26	Deepgram Flux	7.39%	0.021 s	6.5
27	Nemotron 3 ASR (160 ms)	7.61%	0.098 s	—
28	Gladia Solaria	7.82%	1.523 s	12.5
29	Speechmatics Realtime Enhanced	8.05%	0.317 s	17.5
30	Nemotron 3 ASR (80 ms)	8.38%	0.073 s	—
31	Rev.ai	10.14%	0.697 s	3.3

Best Others The best value in each column is filled blue.

Table 17 The three sets in the Artificial Analysis index. The source ranks our two configurations separately; both are shown beside each set’s leading system. source = https://artificialanalysis.ai/speech-to-text/streaming · read 2026-09-09

The three sets in the Artificial Analysis index. The source ranks our two configurations separately; both are shown beside each set’s leading system.
Set	Our configuration	WER	Rank	Leading system on the set	Its WER
VoxPopuli	U3.5 RT Pro — min latency	2.06%	3 of 31	Gemini 3.5 Transcribe Live	1.76%
VoxPopuli	U3.5 RT Pro — max accuracy	2.09%	4 of 31	Gemini 3.5 Transcribe Live	1.76%
AA-AgentTalk	U3.5 RT Pro — min latency	3.53%	9 of 31	Qwen3 ASR Flash Realtime	2.49%
AA-AgentTalk	U3.5 RT Pro — max accuracy	3.48%	7 of 31	Qwen3 ASR Flash Realtime	2.49%
Earnings22	U3.5 RT Pro — min latency	6.96%	11 of 31	Muse Voice Transcribe	4.31%
Earnings22	U3.5 RT Pro — max accuracy	7.08%	12 of 31	Muse Voice Transcribe	4.31%

1 ElevenLabs Scribe v2 Realtime · 0.141 s · 3.59%
2 Gemini 3.5 Transcribe Live · 0.395 s · 4.00%
3 Google Chirp 3 Streaming · 1.276 s · 4.80%
4 OpenAI GPT Realtime (Whisper) · 0.688 s · 4.89%
5 Nemotron 3 ASR (560 ms) · 0.252 s · 5.96%

Figure 8 AA-WER against time to final transcript. The dashed line is the accuracy–latency frontier. source = https://artificialanalysis.ai/speech-to-text/streaming

Coval — Live STT Leaderboard

7-day window, read 2026-09-28 (snapshot generation 1045). The source’s default view is a rolling 24 hours; this is the wider window. WER is pooled over seven datasets — PipeCat production speech (897 clips) and six WildASR conditions (accents 845 clips, clean, clipping, far-field, noise gaps and phone codec at 284 each). Each paired WildASR condition reuses the clean baseline’s utterance, so a difference isolates the condition. Latency is the per-model average. The PipeCat references are model-generated rather than human-verified, which the source notes makes that set better for comparing models than for reading an absolute error rate. Universal-3.6 Pro’s three figures were supplied ahead of the source’s own publication of that model; its placings are computed by inserting them into the 2026-09-28 ranking rather than read from it, and should be re-read once the source publishes. Coval sells voice-agent evaluation tooling.

Table 18 Selected rows of 30. Shown as published, except Universal-3.6 Pro, whose figures precede the source’s publication of that model. Three AssemblyAI models are listed. source = https://benchmarks.coval.ai/stt · read 2026-09-28 · 7-day window

Selected rows of 30. Shown as published, except Universal-3.6 Pro, whose figures precede the source’s publication of that model. Three AssemblyAI models are listed.
System model / snapshot	WER pooled %
AssemblyAI Universal 3.6 Pro rank 1 of 30	2.30%
AssemblyAI Universal 3.5 Pro rank 2 of 30	2.48%
Nari Labs Qwen3 ASR Fast rank 3 of 30	2.72%
Reson8 resonant-1 rank 4 of 30	2.76%
Baseten Qwen3 ASR 1.7b rank 5 of 30	2.78%
Gemini 3.5 Transcribe Live rank 6 of 30	3.01%
ElevenLabs Scribe v2 Realtime rank 17 of 30	4.47%
Soniox STT RT v5 rank 19 of 30	4.80%
Deepgram Flux rank 22 of 30	5.77%
AssemblyAI Universal Streaming rank 23 of 30	5.78%

Best Others The best value in each column is filled blue.

Figure 9 Universal-3.5 Pro’s WER by recording condition — the source publishes no per-condition breakdown for Universal-3.6 Pro. The dashed line is Universal-3.5 Pro’s own pooled value from Table 18. source = https://benchmarks.coval.ai/stt

Pipecat / Daily — STT benchmark

1,000 samples from the smart-turn-data-v3.1-train dataset; the source does not describe the set as English. Pooled WER is total errors over total reference words, judged semantically; TTFS is the median and P95 from when the speaker stops to the final transcription segment. Reproduced with the source’s own model names; ranked here by Pooled WER, where the source now lists rows alphabetically by vendor. Read from the repository README (last changed 2026-09-18). The source also publishes a per-sample WER mean, a transcript count, a perfect-transcript count and a P99, which are not carried here.

Table 19 Shown as published. Three AssemblyAI entries are listed; the source marks universal-3-6-pro as the current one and the other two as superseded. source = https://github.com/pipecat-ai/stt-benchmark · read 2026-09-28

Shown as published. Three AssemblyAI entries are listed; the source marks universal-3-6-pro as the current one and the other two as superseded.
#	System	Pooled WER %	TTFS median ms	TTFS p95 ms
1	Meta muse-voice-transcribe-1.0	0.83%	392	1292
2	AssemblyAI universal-3-6-pro	0.96%	307	401
3	Speechmatics linden-1	1.05%	369	438
4	Speechmatics	1.07%	495	676
5	Azure	1.18%	1016	1345
6	AssemblyAI universal-3-5-pro	1.22%	282	354
7	Cartesia ink-2	1.25%	299	328
8	Soniox stt-rt-v5	1.27%	260	305
9	Soniox stt-rt-v4	1.29%	249	281
10	Deepgram nova-3-general	1.62%	247	298
11	AWS	1.75%	1136	1527
12	NVIDIA Nemotron 3.0 ASR (en)	1.95%	221	238
13	Google gemini-3.5-transcribe-live	2.24%	458	532
14	Smallest AI pulse	2.37%	398	533
15	OpenAI gpt-realtime-whisper	2.73%	740	878
16	Google latest-long	2.85%	878	1155
17	AssemblyAI universal-streaming-english	3.02%	256	362
18	OpenAI gpt-4o-transcribe	3.06%	637	965
19	ElevenLabs scribe_v2_realtime	3.12%	281	348
20	Gradium default	3.71%	570	596
21	Cartesia ink-whisper	4.36%	266	364
22	NVIDIA Nemotron 3.5 ASR (multilingual)	4.58%	236	253
23	Mistral voxtral-mini-transcribe-realtime-2602	4.97%	525	973

Best Others The best value in each column is filled blue.

Cekura — Converse-STT benchmark

Cekura Bench (Converse-STT), last updated 2026-09-16. Accuracy is WER — every replaced, missing and added word against the reference — on two sets: 1,000 short public clips from the Pipecat set (999 for two systems, 998 for one) and 8 licensed recordings from 4 real phone conversations. Latency is the median over 206 streamed turns from the real calls, using each provider’s own finalization method: time to first text and time to final text. Two systems return text only at end of speech and have no first-text time, so first text ranks 17 systems rather than 19. Rows are in the source’s order, by clean-clip WER. The source notes that seven other systems fall within the 95% range of the leading clean-clip WER, so the top of that column is not a clear separation; on the real-call set Universal 3.6 Pro ranks 14th of 19. Cekura sells voice-agent testing and evaluation tooling.

Table 20 Shown as published, all 19 systems. Two AssemblyAI models are listed; Universal 3.6 Pro is the current one. source = https://benchmarks.cekura.ai/stt · read 2026-10-14

Shown as published, all 19 systems. Two AssemblyAI models are listed; Universal 3.6 Pro is the current one.
#	System	WER clean clips, %	WER real calls, %	First text median ms	Final text median ms
1	AssemblyAI Universal 3.6 Pro	1.77%	4.50%	485	91
2	AssemblyAI Universal 3.5 Pro	1.93%	4.23%	489	180
3	Zoom Scribe Pro	1.96%	4.84%	1700	162
4	Zoom Scribe	1.97%	4.88%	1670	118
5	Google Chirp 3	2.00%	4.19%	4480	475
6	Reson8	2.10%	2.93%	2400	235
7	OpenAI GPT Realtime Whisper	2.16%	3.53%	1380	543
8	Cartesia Ink 2	2.39%	3.09%	1510	123
9	Speechmatics Linden	2.46%	3.93%	1190	147
10	Soniox STT-RT v5	2.54%	3.92%	1200	38
11	Google Chirp 2	2.59%	3.80%	4500	568
12	Inworld STT-1	2.72%	10.60%	1060	41
13	OpenAI GPT-4o Mini Transcribe	3.35%	4.45%	at speech end	630
14	Deepgram Nova-3	3.46%	4.39%	1060	103
15	Smallest Pulse	3.66%	3.46%	1060	201
16	OpenAI GPT-4o Transcribe	3.90%	12.45%	at speech end	652
17	Deepgram Flux English	3.90%	4.18%	791	108
18	Deepgram Flux Multilingual	4.96%	4.53%	793	114
19	Gradium	6.48%	4.45%	1710	212

Best Others The best value in each column is filled blue.

09How we measure

How the results were produced: the harness, the metrics, why public leaderboards can rank the same systems differently, and the limits of this report.

How we run every system

Protocol. Audio is streamed in real time to each system’s public API, default settings, language fixed. One normalizer is applied before scoring. Each streaming model a vendor ships has its own row.

Complete runs only. An average is shown only when the system was scored on every dataset in it; partial rows keep their cells but are not ranked.

n/d, n/s. A 0.00 error rate means no data. WER or CER of 60% or more means the language is unsupported; the cell is excluded from means.

Excluded. Unreleased models, and results on a superseded version of the voice-agent test set.

Systems

Every system with a public streaming API and a current run; a dot marks each suite it was scored on.

Table 21 Systems evaluated and the suites each has a current run on. 5 suites · runs 2026-08-27 → 2026-09-22 · added this update: Meta, Zoom

Systems evaluated and the suites each has a current run on.
Vendor	Model	English / CI	Voice agents	Code-switch.	Diarization	Multilingual	Runs
AssemblyAI	Universal-3.5 Pro Realtime	●	●	●	●	●	2026-08-18 → 2026-09-02
AssemblyAI	Universal-3.6 Pro Realtime	●	●	●	●	●	2026-09-03 → 2026-09-21
AWS	Transcribe Streaming	●	●	●	●	●	2026-08-19 → 2026-08-31
Azure	Azure Realtime STT	●	●	●	●	●	2026-08-19 → 2026-09-11
Cartesia	Ink 2 (turns)	·	·	·	·	·	2026-08-31
Cartesia	Ink-Whisper	●	●	·	✕	●	2026-08-19 → 2026-08-30
Cartesia	Ink-Whisper EN	·	·	●	✕	·	2026-08-20
Deepgram	Flux	·	·	·	✕	●	2026-08-25
Deepgram	Flux EN	●	●	●	✕	·	2026-08-19 → 2026-09-11
Deepgram	Flux Multi	·	●	·	✕	·	2026-08-27
Deepgram	Nova-3	●	●	·	●	●	2026-08-18 → 2026-08-31
Deepgram	Nova-3 EN	·	·	●	·	·	2026-08-20
Deepgram	Nova-3 Multi	·	·	●	·	·	2026-08-20
ElevenLabs	Scribe v2 Realtime	●	●	●	✕	●	2026-08-19 → 2026-08-31
Gladia	Solaria	●	●	●	✕	●	2026-08-19 → 2026-09-10
Google	Gemini 3.5 Transcribe Live	●	●	●	✕	●	2026-08-31 → 2026-09-06
Meta	Muse Voice Transcribe (new)	●	●	●	●	●	2026-09-03 → 2026-09-15
Mistral	Voxtral Mini Realtime	●	●	●	✕	●	2026-08-19 → 2026-08-31
OpenAI	GPT Realtime	●	·	●	✕	●	2026-08-19 → 2026-08-31
OpenAI	GPT-4o mini Transcribe	●	●	●	✕	●	2026-08-19 → 2026-08-31
OpenAI	GPT-4o Transcribe	●	●	●	✕	●	2026-08-19 → 2026-08-31
Reson8	Reson8	●	●	●	●	●	2026-09-01 → 2026-09-18
Smallest	Pulse	●	●	●	●	●	2026-08-28 → 2026-08-31
Soniox	Soniox Realtime	●	●	●	●	●	2026-08-31 → 2026-09-23
Speechmatics	Enhanced Realtime	●	●	●	●	●	2026-08-19 → 2026-08-31
Speechmatics	Standard Realtime	●	●	●	●	●	2026-08-19 → 2026-08-31
xAI	Grok Streaming	●	●	●	●	●	2026-08-19 → 2026-09-14
Zoom	Scribe (Live) (new)	●	●	●	✕	●	2026-09-10 → 2026-09-16

Test data

The audio each benchmark uses and the metric it scores. Each results section also lists its own datasets.

Table 22 Test data by benchmark. Files are scored recordings or clips; multilingual, code-switching and judge-scored sets are scored per language or label, so no single count applies. Public corpora are named, licensed sets described.

Test data by benchmark. Files are scored recordings or clips; multilingual, code-switching and judge-scored sets are scored per language or label, so no single count applies. Public corpora are named, licensed sets described.
Suite	Audio	Files	Scored on
Voice agents	Scripted caller scenarios voiced by real speakers, recorded in three noise environments, three utterance-length bands and 12 speaker groups	12,460	Entity error, WER
Healthcare	PriMock57 primary-care consultations; licensed clinician dictation; respiratory-clinic recordings	479	Medical entity error
English short-form	Common Voice; LibriSpeech test-clean and test-other	21,588	WER
English accented	British-, French-, Spanish- and Indian-accented English	9,315	WER
English long-form	TED-LIUM talks; Rev16 podcasts; CALLHOME telephone calls	91	WER
Judge-scored entities	Short-form English audio, with entities graded per label by an LLM judge	—	Entity error (judge)
Non-speech	Hold music, line noise and silence with no speech	2,950	Response rate
Multilingual	FLEURS and Common Voice test sets in eighteen languages	—	WER (character error rate for Japanese, Chinese)
Code-switching	Five English↔X pairs derived from read speech, plus a bilingual conversational corpus	—	WER
Diarization	CALLHOME calls; AMI and NOTSOFAR meetings; DiPCo dinner parties	203	DER, cpWER

Each test set is marked by how its audio can be obtained: ○ public and open source; ◆ public but licensed or purchasable; ● private, recorded or annotated by us.

How we score

Each metric and how it is computed. The same normalizer and scoring code apply to every system.

WER Word error rate after normalization; lower is better. Character error rate for Japanese, Chinese and Korean.

NEER Normalized entity error rate: the share of entities (names, addresses, numbers, emails, dates, codes, medical and technical terms) transcribed incorrectly.

MER Medical entity error rate over medication and disease terms.

Judge-scored entity error Entity error rate as graded by an LLM judge.

DER / cpWER Diarization error rate, and concatenated minimum-permutation WER.

Non-speech response rate Share of non-speech clips for which the system produces any text.

Streaming harness. Audio is chunked and streamed in real time to each vendor’s endpoint through its documented SDK or WebSocket API, default settings, language fixed. Final transcripts are scored, not partials.

Normalization. One normalizer (case, punctuation, numbers, abbreviations) is applied to references and to every system’s output. Entity metrics are computed on normalized text and, where reported, on formatted text.

Entity error. Entity spans are located in the reference; an entity is an error if any token of its span is wrong in the aligned transcript. The judge-scored variant asks an LLM whether the transcript names the same real-world entity.

Selection rule. Every row is the model’s most recent production run. The overview grid keeps each vendor’s best model per column and names it on hover.

Provenance. Each figure carries its metric and run-date span. Public test sets are named; licensed sets are described.

Why public leaderboards and ours differ

We build for voice agents, clinical documentation, meetings, telephony, and multilingual and code-switched speech, so the test sets cover that range of speakers, accents, languages and acoustic conditions and score what matters in each. Public leaderboards usually ask a narrower question, typically one word-error-rate figure on a few corpora, so a model can rank differently on the two.

Table 23 What each benchmark measures: this report and the public leaderboards reproduced in §9.

What each benchmark measures: this report and the public leaderboards reproduced in §9.
Benchmark	Audio	Scores	Run by	References
This report	Voice-agent calls in three noise environments, clinical consultations and dictation, accented and long-form English, eighteen languages, code-switched speech, meetings and telephone calls, non-speech audio	Entity error (by class), WER, DER, non-speech response rate	AssemblyAI; every vendor on default settings	Human-verified
Artificial Analysis	AA-AgentTalk (50%), VoxPopuli parliamentary speech (25%), Earnings22 calls (25%)	One WER index over the three sets; time to final transcript	Artificial Analysis	Published with the sets
Coval	Short clips under seven recording conditions (clean, accent, far-field, phone codec, reverb…), 7-day window	Pooled WER; time to first token; time to final segment	Coval	Model-generated
Pipecat / Daily	1,000 English samples	Semantic WER; median and P95 time to final	Daily	Published with the set
Cekura	1,000 short public clips from the Pipecat set; 8 licensed recordings from 4 real phone calls	WER on each set; median time to first text and to final text	Cekura	Not stated by the source

The Artificial Analysis index is one WER over three sets, half of it AA-AgentTalk, with no entity, noise, medical, multilingual, code-switching or diarization component; its latency runs from a voice-activity detector’s end of speech to the final transcript, which this report does not measure. Coval scores against model-generated references. Neither substitutes for the other or for this report.

Caveats

We ran it. AssemblyAI produced the results in §1–§8. Competitors ran on default settings and may do better tuned. §9 has independent results.

Snapshots differ. Runs span 2026-08-18 to 2026-09-24 (Table 21). A vendor that updated its model within that window may appear on the older version.

Coverage is uneven. Not every system ran every set; means are withheld rather than computed on partial data.

Reference quality. TED-LIUM references omit sponsor read-outs. Some independent leaderboards (Coval) use model-generated references.

Benchmarks

Run your own benchmark

Testing on your own audio? Want to do it right? Read our benchmarking guide in the docs.

Learn more Get a custom benchmark

AssemblyAI Universal-3.6 Pro Realtime benchmarks

Pre-recorded speech recognition, measured against the field

01English

Clean transcription

Verbatim transcription

Hear it for yourself

Accented speech

02Named Entities

03Diarization

04Code switching

05Multilingual

06How we measure

How the runs were made

Systems

Test data

How we score

Run your own benchmark

Realtime speech recognition, measured against the field

01Overview

Position in the field

02Voice agents

Background noise and utterance length

Listen to the test data

Speaker groups

Ablations on our own model

Word error rate

Data

03Healthcare

Overall

Medication and disease

Data

04English

Word error rate — short-form

Listen to the test data

Word error rate — accented English

Word error rate — long-form

Data

05Multilingual

FLEURS — Systems covering all 18 languages

FLEURS — Systems covering fewer than 18

Common Voice — Systems covering all 18 languages

Common Voice — Systems covering fewer than 18

Data

06Code-switching

Data

07Diarization

Data

08Independent evaluations

Artificial Analysis — Speech to Text (Streaming)

Coval — Live STT Leaderboard

Pipecat / Daily — STT benchmark

Cekura — Converse-STT benchmark

09How we measure

How we run every system

Systems

Test data

How we score

Why public leaderboards and ours differ

Caveats

Run your own benchmark

Methodology