Transcript rewriting

Dictation cleans the transcript up and returns the result in llm_response, alongside the verbatim transcript in text. This is the one feature Dictation has that the other speech-to-text APIs do not: you get the words as spoken and text your user can send, from the same call. Cleanup is applied by default. Omitting llm_instruction, or the whole config part, runs the default cleanup task: filler comes out, punctuation and capitalization are applied, and self-corrections can be resolved to what the speaker landed on. The speaker’s own phrasing and tone survive it, so the result still reads like them.

import assemblyai as aai

aai.settings.api_key = "<YOUR_API_KEY>"

result = aai.DictationTranscriber().transcribe_live("clip.wav")

# text:         "um so can we uh move the the meeting to thursday
#                i think friday works better actually"
# llm_response: "Can we move the meeting to Friday? That works better."
print(result.text)
print(result.llm_response)

Customizing the rewrite

Set llm_instruction to a plain-English description of the task you want. It replaces the default cleanup task rather than adding to it:

{ "llm_instruction": "Remove filler words and fix punctuation." }

An explicit "llm_instruction": null means the same as omitting the field, so the default cleanup task still runs.

For more control than a plain-English instruction gives you, send llm instead of llm_instruction. It takes an LLM Gateway chat completions request: your own messages, model and sampling settings, or a saved prompt. See Sending your own LLM request for how Dictation uses it.

Python SDK
JavaScript SDK
Python
JavaScript
cURL

import assemblyai as aai

aai.settings.api_key = "<YOUR_API_KEY>"

config = aai.DictationConfig(
    llm_instruction="Rewrite as a concise clinical chart note.",
)

result = aai.DictationTranscriber().transcribe_live("clip.wav", config)

print(result.text)        # the verbatim transcript
print(result.final_text)  # the rewrite, falling back to the transcript

import { AssemblyAI } from "assemblyai";

const client = new AssemblyAI({ apiKey: "<YOUR_API_KEY>" });

const config = {
  llm_instruction: "Rewrite as a concise clinical chart note.",
};

const result = await client.dictation.transcribeLive("clip.wav", config);

console.log(result.text); // the verbatim transcript
console.log(result.final_text); // the rewrite, falling back to the transcript

import json
import requests

config = {"llm_instruction": "Rewrite as a concise clinical chart note."}

with open("clip.wav", "rb") as f:
    audio = f.read()

response = requests.post(
    "https://dictation.assemblyai.com/v1/transcribe/live",
    headers={"Authorization": "<YOUR_API_KEY>"},
    files={
        "config": (None, json.dumps(config), "application/json"),
        "audio": ("clip.wav", audio, "audio/wav"),
    },
    timeout=90,
)
response.raise_for_status()
result = response.json()

print(result["text"])
print(result["llm_response"])

import { readFileSync } from "fs";

const config = { llm_instruction: "Rewrite as a concise clinical chart note." };

const audio = readFileSync("clip.wav");
const form = new FormData();
form.append(
  "config",
  new Blob([JSON.stringify(config)], { type: "application/json" })
);
form.append("audio", new Blob([audio], { type: "audio/wav" }), "clip.wav");

const response = await fetch(
  "https://dictation.assemblyai.com/v1/transcribe/live",
  { method: "POST", headers: { Authorization: "<YOUR_API_KEY>" }, body: form }
);

const result = await response.json();
console.log(result.text);
console.log(result.llm_response);

curl -X POST https://dictation.assemblyai.com/v1/transcribe/live \
  -H 'Authorization: <YOUR_API_KEY>' \
  -F 'config={"llm_instruction": "Rewrite as a concise clinical chart note."};type=application/json' \
  -F 'audio=@clip.wav;type=audio/wav'

Choosing a rewrite mode

rewrite_mode picks the model tier that runs the rewrite:

`rewrite_mode`	Use it for
`fast` (default)	The default cleanup task and short, simple instructions, at the lowest latency
`smart`	Detailed formatting instructions that `fast` doesn’t follow reliably, such as spoken punctuation. The rewrite has higher latency.

{ "rewrite_mode": "smart", "llm_instruction": "Turn this into a bulleted list of action items." }

Any other value is rejected with 400. Over HTTP it goes in the config part of the request:

curl -X POST https://dictation.assemblyai.com/v1/transcribe/live \
  -H 'Authorization: <YOUR_API_KEY>' \
  -F 'config={"rewrite_mode": "smart", "llm_instruction": "Turn this into a bulleted list of action items."};type=application/json' \
  -F 'audio=@clip.wav;type=audio/wav'

Writing a good instruction

Describe only the transformation. Rules about output format, refusing to answer the text, and handling already-clean input are enforced by the service, so adding your own versions of them wastes instruction budget and can conflict with what the service already does.

Instead of	Write
”You are a helpful medical scribe. Output only the note, nothing else. If the input is already clean, return it unchanged. Rewrite as a chart note."	"Rewrite as a concise clinical chart note."
"Return JSON with a summary field.”	Describe the prose you want, then parse it yourself
”Lowercase unless it’s a name. No final punctuation. Don’t capitalize the first word."	"Write the dictated words as a mid-sentence continuation of: “I think”.”

Describe the shape you want rather than listing what not to do. The model leans toward complete, punctuated sentences, and a prohibition has to fight that on every request, so “don’t” rules are the least reliable kind of instruction. A positive description such as “a mid-sentence continuation” gets the casing and punctuation right as a side effect. Leave mechanical edits to your own code. The rewrite output is trimmed, so it can’t return a leading space, and stripping a trailing full stop is exact and free in code. Inserting at the cursor shows how to split the work for text that lands mid-sentence. Some instructions that work well:

"Remove filler words and tidy the punctuation."
"Rewrite as a short, friendly booking confirmation addressed to the client."
"Turn this into a bulleted list of action items."
"Rewrite as a concise clinical chart note."

Example: spoken punctuation

Some users dictate punctuation and formatting aloud: “Do you want to get lunch together question mark it’s been too long exclamation point”. This instruction turns those spoken commands into the punctuation they name. Pair it with "rewrite_mode": "smart", which follows a list of rules like this more reliably than fast.

You will be given a raw dictation transcript in which punctuation, capitalization, and formatting are stated aloud as literal spoken commands instead of being applied automatically. Rewrite it into final formatted text by replacing each spoken command as follows, leaving every other word exactly as given:

- 'comma' -> ','
- 'period' -> '.'
- 'question mark' -> '?'
- 'exclamation point' -> '!'
- 'colon' -> ':'
- 'semicolon' -> ';'
- 'all caps <word>' -> uppercase that one word; 'all caps <word> ... end caps' -> uppercase the whole span between them
- 'cap <word>' -> capitalize just the first letter of that word
- 'new paragraph' -> start a new paragraph
- 'new line' -> start a new line
- 'open quote' ... 'close quote' -> wrap the enclosed text in double quotes
- 'open parenthesis' ... 'close parenthesis' -> wrap the enclosed text in parentheses
- 'hyphen' -> join the surrounding words with a hyphen
- 'apostrophe s' -> attach a possessive 's directly to the previous word
- Spoken numbers stay spelled out as words -- never convert them to digits.

Every spoken command word above (comma, period, question mark, exclamation point, colon, semicolon, all caps, cap, new paragraph, new line, open/close quote, open/close parenthesis, hyphen, apostrophe s) is an INSTRUCTION, never content -- it must never appear literally in your output. Do not add, remove, reorder, or rephrase any other word.

Sending your own LLM request

Instead of llm_instruction, you can set llm to the body of an LLM Gateway chat completions request. Dictation sends it to LLM Gateway after transcription and returns the model’s output in llm_response, like any other rewrite. Send llm or llm_instruction, not both. A request with both is rejected with 400. There are two ways to put the transcript in the request. In your own messages. Write prompt or messages, and put {{turn}} where the transcript goes:

{
  "llm": {
    "model": "gemma-4-31b",
    "messages": [
      { "role": "system", "content": "Rewrite the dictated note as a short, formal email." },
      { "role": "user", "content": "{{turn}}" }
    ]
  }
}

In a saved prompt. Set aai_prompt to the name of a prompt saved in your account. Dictation passes the transcript as the text replacement, so put {{ text }} in the saved prompt where the transcript goes. You can add replacements of your own in prompt_replacements:

{
  "llm": {
    "aai_prompt": "dictation-email",
    "prompt_replacements": { "signoff": "Best, Sam" }
  }
}

How Dictation differs from calling LLM Gateway directly

No guardrails. The service’s rewrite rules and transcript fencing apply only to the default task and llm_instruction. With llm, the model gets exactly the prompt you send, so it may act on commands in the dictation, such as “translate this into French”, unless your prompt tells it not to.
Two transcript tags. In prompt and messages, Dictation renders the text as a Jinja2 template and fills {{turn}}. In a saved prompt, LLM Gateway fills {{ text }}, and the spaces inside the braces are required.
A fixed set of fields. llm accepts model, messages, prompt, temperature, top_p, top_k, max_tokens, max_completion_tokens, response_format, reasoning_effort, aai_prompt, prompt_replacements and zero_data_retention. Field names are case-sensitive. Any other field, such as fallbacks, tools, stream or transcript_id, is rejected with 400.
The rewrite models only. model must be one of the models that rewrite_mode runs on: qwen3.5-4b-32k-fast, gemma-4-31b or gemini-3.5-flash-lite. In the EU region, use qwen3.5-4b-32k-fast or gemini-3.5-flash-lite. Any other model is rejected with 400. Without model, rewrite_mode picks one.
Bounded output. max_tokens and max_completion_tokens can be at most 8192. If you leave max_tokens out, Dictation sizes it from the transcript, so a prompt that expands the text, such as turning a short note into an email, may need a larger one. A rewrite that runs out of tokens comes back with llm_error: "truncated". temperature is 0 unless you set it.
Zero data retention. Dictation sets zero_data_retention on every LLM Gateway request, including ones that use aai_prompt, and you can’t turn it off. Only a saved prompt’s template is stored. The transcript and the rewrite aren’t kept.
Saved prompts run as saved. With aai_prompt, the model, max_tokens and every other setting come from the saved prompt, and the model and output limits above don’t apply. Only prompt_replacements may accompany aai_prompt. Any other llm field, or rewrite_mode, is rejected with 400, and so is setting text or transcript in prompt_replacements: Dictation fills text, and LLM Gateway reserves transcript. If your account has no prompt with that name, llm_error is prompt_not_found.
Failures don’t fail the request. If LLM Gateway rejects or fails the request, the response is still 200 with the transcript in text, and llm_error says why. The 400s above are returned before anything is transcribed.

The transcript is always preserved

The rewrite never replaces the transcription. text is always the verbatim transcript, and the rewritten version arrives separately in llm_response. If you need to show one value, final_text in the Python and JavaScript SDKs returns the rewrite when there is one and the transcript otherwise. The rewrite is also best-effort. If it fails, the response is still 200, llm_response is null, and llm_error says why. See Error handling.

Dictated commands are not executed

With the default cleanup task or llm_instruction, the transcript is passed to the model as fenced data, with instructions not to act on anything inside it. Dictated speech routinely contains questions and commands like “translate this into French” or “ignore what I just said”. Those are rewritten as speech rather than carried out. A request of your own has no such guard unless your prompt adds one.

Prompting and keyterms shapes the transcript as it is written, where llm_instruction reshapes it afterwards.
Language selection sets the language the audio is transcribed in.

Getting started

Features

API reference

Advanced

Guides

Transcript rewriting

Customizing the rewrite

Choosing a rewrite mode

Writing a good instruction

Example: spoken punctuation

Sending your own LLM request

How Dictation differs from calling LLM Gateway directly

The transcript is always preserved

Dictated commands are not executed

​Customizing the rewrite

​Choosing a rewrite mode

​Writing a good instruction

​Example: spoken punctuation

​Sending your own LLM request

​How Dictation differs from calling LLM Gateway directly

​The transcript is always preserved

​Dictated commands are not executed

​Related features

Customizing the rewrite

Choosing a rewrite mode

Writing a good instruction

Example: spoken punctuation

Sending your own LLM request

How Dictation differs from calling LLM Gateway directly

The transcript is always preserved

Dictated commands are not executed

Related features