AssemblyAI vs Deepgram for medical transcription

Both of these APIs will transcribe a clinical encounter competently. That's not the interesting part. The interesting part is what happens to the fifteen or twenty words per encounter that actually carry clinical meaning — the drug names, the dosages, the procedure references — because those are the words that end up in a note, feed a summarizer, and occasionally cause someone to chart the wrong medication.

On our published medical benchmarks, Universal-3.5 Pro with Medical Mode records a 3.2% Missed Entity Rate. That is the lowest across benchmarked providers. The number looks small until you multiply by the number of clinical entities in a fifteen-minute visit and then by the number of visits a health system runs in a day.

This is a head-to-head on the dimensions that actually differ: entity accuracy, how each system handles context, diarization for multi-party consultations, pricing structure, and what migration looks like. We're AssemblyAI, so read accordingly — but every figure here is one we publish and one you can reproduce on your own audio.

The short version

Missed entity rate is the number to compare on

If you're benchmarking these two on word error rate, stop. WER treats every token as equal weight, so "the" and "amiodarone" count the same. A model can post a strong overall WER while concentrating its errors precisely in the medication names — which is the failure mode that matters and the one WER is structurally blind to.

Missed Entity Rate scores only the clinically meaningful tokens: drugs, dosages, conditions, procedures, anatomy. It's the metric we publish for Medical Mode because it's the metric that predicts whether a clinician has to rework a note.

Against the base model without Medical Mode, enabling it gives roughly 20% fewer missed medical entities and 87% fewer entity errors. Against the field, 3.2% MER is the lowest across the providers we benchmarked — Deepgram, Speechmatics, AWS, and Google. Methodology and the full table are on our benchmarks page.

Run your own eval anyway

We mean this. Published benchmarks tell you which models are in contention; your audio tells you which one wins. The exercise takes two days:

• Pull 25 to 50 real encounters that represent your actual mix — specialties, accents, mic setups, room noise.
• Build a ground-truth entity list from the notes your clinicians wrote for those encounters.
• Run both APIs with their clinical modes on and score entity capture, not WER.
• Score diarization separately, and score it on short turns specifically.

If a vendor's advantage disappears on your audio, that's the most valuable thing you'll learn in the whole procurement cycle.

What the errors actually look like

Abstract percentages don't convey the risk profile. The concrete version:

Sound-alike drug pairs. Hydralazine and hydroxyzine. Clonidine and Klonopin. Metoprolol and metronidazole. These are separated by one or two phonemes, and those phonemes are the first casualties when a clinician talks fast over a running sink. A general model resolves the ambiguity toward whichever word is more common in everyday speech, which is usually the wrong one.

Dosage mangling. "Twenty-five milligrams BID" becoming "twenty-five milligrams by day," or a decimal shifting. Numbers are where formatting decisions in the model become clinical decisions in the chart.

Negation loss. "No history of MI" losing the "no." This is the error class that scares clinical safety reviewers the most because it inverts meaning while looking perfectly fluent.

Attribution flips. A patient's report of a symptom attributed to the clinician, or a family member's speculation recorded as patient history.

A clinical model reduces the first three by shifting the model's decoding priors toward medical language. The fourth is a diarization problem, covered below.

Context handling: free text vs a keyword list

This is the architectural difference that has the biggest downstream consequence, and it rarely shows up in a feature comparison.

Deepgram's approach to specialist terminology is keyterm prompting — you pass a list of terms you expect. That works, and it's a genuine improvement over nothing. The cost is operational: someone has to build, maintain, and version those lists. Per specialty. Per formulary. Per customer, if you're a vendor. Forever.

AssemblyAI's contextual prompting takes free text instead. You hand the model prose describing what it's about to hear, and it adapts. In a clinical setting the highest-value prose is already sitting in your system: the patient's prior-visit note. In an internal healthcare test, feeding exactly that cut missed medical terms by 31%.

The difference between those two designs is the difference between maintaining a dictionary and passing a pointer to data you already have. A patient on lisinopril, metformin, and atorvastatin who was seen six weeks ago for the same complaint has a vocabulary — you don't need to curate it, you need to hand it over.

On the streaming side, the equivalent is agent_context, which cut WER 10.2% across 20,000 voice agent files, and detailed context cut medical-term entity errors 43%. Details in the Universal-3.5 Pro Realtime writeup.

Diarization for multi-party consultations

Most clinical encounters have more than two participants. A caregiver, an interpreter, a resident, a parent answering for a child. And most diarization benchmarks are scored on diarization error rate, which measures how much audio landed under the right speaker — a metric that rewards getting long monologues right and quietly forgives missing every short interjection.

Short interjections are the clinical content. "Since when?" "About a week." "Any chest pain with it?" "No."

Universal-3.5 Pro's diarization is optimized for cpWER, which only improves if the correct words land under the correct speaker. It's the most accurate diarization we've shipped, and the gains concentrate on short turns and overlapped speech — exactly the audio profile of a real consultation. In streaming, labels support revision as more audio arrives, up to 10 speakers, so an early guess gets corrected rather than baked in.

If you're evaluating both vendors on diarization, score it on short turns specifically. Take the utterances under five words and check attribution on those alone. That's the number that will differ.

Multilingual clinical audio

Universal-3.5 Pro code-switches natively across 18 languages with no configuration — a patient moving between English and Spanish mid-sentence doesn't need upstream language detection or routing.

Medical Mode's clinical entity layer covers English, Spanish, German, and French, for both pre-recorded and streaming audio. Keep those two facts separate when you're evaluating: the base transcription handles considerably more languages than the medical entity layer does. If your patient population is primarily in one of those four, you get both. If it isn't, you still get accurate general transcription, just without the clinical entity lift.

Pricing and how the bill actually works

AssemblyAI's medical pricing is two numbers you can add:

• Async: $0.21/hr for Universal-3.5 Pro, plus $0.15/hr for Medical Mode. $0.36/hr combined.
• Streaming: $0.45/hr base, plus $0.15/hr for Medical Mode. $0.60/hr combined.
• Voice agents: flat $4.50/hr for the Voice Agent API, one WebSocket replacing STT, LLM, and TTS.

Current rates are always on pricing. What we'd flag as the thing to actually compare isn't the headline rate — it's the structure. Ask both vendors these three questions:

1. Is the clinical mode a surcharge on the flagship model, or does it route you to a different model with a different accuracy profile?
2. What does the annual commit look like, and what happens if you undershoot it?
3. What's the effective rate at your projected volume, including any minimum?

Per-hour list prices are the least important variable in a healthcare contract. Commit structure and overage terms usually matter more.

PHI, BAA, and deployment options

Both vendors will sign a BAA, so this isn't a differentiator on the yes/no question. It's a differentiator on the details.

AssemblyAI signs a Business Associate Addendum (BAA) for customers processing PHI and acts as a business associate under HIPAA for that data. The process is in our BAA FAQ; the agreement is at legal/business-associate-agreement. We're SOC 2 Type 2.

The parts worth comparing line by line with any vendor:

• Redaction scope. Ours covers both audio and transcripts. A redacted transcript next to an unredacted recording isn't a redacted encounter.
• Data residency. EU residency is available at api.eu.assemblyai.com.
• Self-hosting. Available when the audio can't leave your infrastructure at all — the answer for some health system deployments and most defense-adjacent ones.
• Retention and training. Ask explicitly whether your audio is retained and whether it's used for model training. Get it in writing.

Migrating from Deepgram

Practically, this is a small change. You're swapping an HTTP client and a config object, not rebuilding a pipeline.

import assemblyai as aai

aai.settings.api_key = "YOUR_API_KEY"

config = aai.TranscriptionConfig(
    speech_models=["universal-3-5-pro"],
    domain="medical-v1",
    speaker_labels=True,
)

transcript = aai.Transcriber().transcribe("encounter.wav", config=config)

for utterance in transcript.utterances:
    print(f"Speaker {utterance.speaker}: {utterance.text}")

Streaming connects to wss://streaming.assemblyai.com/v3/ws and takes speech_model singular:

from assemblyai.streaming.v3 import (
    StreamingClient,
    StreamingClientOptions,
    StreamingParameters,
)

client = StreamingClient(
    StreamingClientOptions(api_key="YOUR_API_KEY")
)

client.connect(
    StreamingParameters(
        sample_rate=16000, speech_model="universal-3-5-pro",
        domain="medical-v1",
        voice_focus="far-field",
        mode="max_accuracy",
        speaker_labels=True,
    )
)

The migration work that isn't code: retire your keyterm lists and replace them with a context assembly step, and re-tune whatever downstream extraction logic you built to compensate for your previous model's error patterns. That second one usually takes longer than the API swap. Full reference in the docs.

Where the rest of the field lands

For context, since most evaluations aren't actually a two-horse race. Amazon Transcribe Medical is not published in our benchmark — the tradeoff is that it's already inside AWS, which for some teams is decisive. Google Cloud offers medical conversation and dictation models with the same "already in the cloud" logic. Speechmatics has enhanced medical vocabulary. NVIDIA Riva gives you full control and full responsibility for domain training. Nuance Dragon and DAX and Abridge are products rather than APIs — different purchase entirely, and if you're building your own scribe they're your competitors, not your vendors.

We go provider by provider in our roundup of the best medical speech-to-text options, and cover the API selection process more broadly in the medical transcription API guide.

How we'd decide

Honest framing:

Pick Deepgram if you're already deep in their stack, your entity accuracy is measurably good enough on your audio, and the migration cost outweighs a benchmark difference you can't feel in production.

Pick AssemblyAI if entity accuracy is the constraint you keep hitting, if you're tired of maintaining terminology lists, if multi-party diarization on short turns is breaking your notes, or if you want the clinical mode to be a parameter on the flagship model rather than a separate model with its own quirks.

Test both in every case. Two days of eval is cheap against a multi-year contract.

What changes next

The benchmark gap between the leading clinical models is going to keep narrowing — that's what happens to every accuracy race. The durable advantage is shifting to context, and that's a different kind of competition.

Once a model's performance on a given encounter depends materially on what you tell it beforehand, the vendor comparison stops being "whose model is better" and becomes "whose model best exploits the data I already have." A 31% reduction in missed terms from a prior-visit note isn't a model improvement; it's an integration improvement. Teams that build patient-context assembly into their transcription layer now will find their accuracy improving without changing vendors at all — and teams that don't will keep re-running vendor bakeoffs to chase a percentage point they could have gotten from their own database. Choose accordingly. See medical solutions for how this fits together.

Frequently asked questions

AssemblyAI Medical Mode vs Deepgram Nova-3 Medical: which is more accurate for clinical transcription?

On our published medical benchmarks, Universal-3.5 Pro with Medical Mode records a 3.2% Missed Entity Rate, the lowest across benchmarked providers. Medical Mode also delivers 87% fewer entity errors than the base model without it. Verify on your own audio — see benchmarks for methodology.

How accurately does each handle pharmaceutical and drug names?

Drug names are the hardest category because sound-alike pairs differ by one or two phonemes. Medical Mode shifts the model's decoding priors toward clinical language, which is why the entity numbers move so much more than the overall word error rate does. Contextual prompting adds a second layer: passing a patient's prior-visit note cut missed medical terms by 31% in an internal healthcare test.

Can either handle a multi-speaker consultation with a caregiver or interpreter?

Yes, both offer diarization. The difference is the optimization target. Universal-3.5 Pro is optimized for cpWER rather than diarization error rate, so it holds up on short turns and overlapped speech — the two-word exchanges that carry most of the clinical content. Streaming supports up to 10 speakers with label revision.

How does AssemblyAI handle HIPAA and PHI?

AssemblyAI signs a Business Associate Addendum (BAA) for customers processing PHI and acts as a business associate under HIPAA. We're SOC 2 Type 2, offer PHI redaction across both audio and transcripts, and support EU data residency and self-hosted deployment. See the BAA FAQ.

How do the costs compare?

AssemblyAI is $0.36/hr for async with Medical Mode ($0.21 base plus $0.15) and $0.60/hr for streaming with Medical Mode ($0.45 base plus $0.15). Deepgram publishes per-minute rates that vary by tier and commitment. The more important comparison is contract structure — commit size, overage terms, and effective rate at your volume. Our rates are on pricing.

What does migrating from Deepgram involve?

The API swap is a config change — a client, a model ID, and domain: "medical-v1". The real work is retiring your keyterm lists in favor of a context assembly step and re-tuning downstream extraction that was calibrated to your old model's error patterns. Most teams run both in parallel on live traffic for a couple of weeks before cutting over. See medical transcription use cases.

‍