If you’re using Cursor, Claude Code, Copilot, ChatGPT Codex, Devin, or another AI coding assistant to build with AssemblyAI, give your agent live context so it doesn’t rely on outdated training data.
Start here
New to AssemblyAI?
Paste our full integration prompt into your agent. Teaches it operating rules, recommended models, parameter gotchas, and quickstarts for every product surface — so the first code you get back is closer to production-ready.
Already in a project?
Add a one-liner to your AGENTS.md, CLAUDE.md, or .cursorrules so your agent refetches the latest integration prompt before every AssemblyAI prompt — the most effective option and what we recommend for ongoing work.
Claude Code and other skills-compatible agents (60+, via the universal skills format).
You want curated SDK-specific context (Python / JS / streaming / voice agents) bundled with the agent, auto-loaded when relevant.
Installed snapshot; re-install to pick up doc changes.
For most teams, pinning the URL is enough. Add the MCP server on top if your agent does a lot of mid-session lookups. The copy-paste prompt is best kept for first-time use or one-shot chat sessions.
Pin the URL in your agent instructions
For ongoing projects, point your agent at the integration prompt URL from its project instructions. The agent re-fetches the prompt before every AssemblyAI-related prompt, so you automatically pick up API changes without bloating your context window.Add this to your project’s AGENTS.md, CLAUDE.md, .cursorrules, .cursor/rules/, .github/copilot-instructions.md, .windsurfrules, or whichever instructions file your agent reads:
Before writing AssemblyAI code, read https://www.assemblyai.com/docs/agent-instructions.mdand https://www.assemblyai.com/docs/llms.txt. The API has changed — do not rely onmemorized parameter names.
The two URLs cover different things:
agent-instructions.md — the rules of the road: operating rules, discovery questions, recommendation template, feature selection, quickstarts for every product surface, error handling, and the gotchas that trip up most integrations.
llms.txt — the documentation index: every page’s title, URL, and one-line description so the agent can navigate to detail pages.
For deep lookups, llms-full.txt returns the full concatenated documentation content. It is too large for most agents to read in one fetch, so llms.txt also links section files under 100 KB each; fetch only the sections you need.
Copy the integration prompt
For a one-shot agent session — a single ChatGPT Codex run, a quick Cursor chat without project files, or just trying AssemblyAI for the first time — paste the full integration prompt into your agent along with a short description of what you want to build. The prompt teaches the agent the operating rules, discovery questions, recommended models, parameter gotchas, and quickstarts for every product surface, so the first code you get back is closer to production-ready than what the agent would write from memory.For ongoing project work, pin the URL instead — pasting the full prompt into your project instructions stuffs every message with ~770 lines of context.
View and copy the full integration prompt
# AssemblyAI Integration — Coding Agent InstructionsYou are helping a developer integrate AssemblyAI's Speech-to-Text API into their application. Your job is to understand their context through discovery, produce a concrete implementation plan, get their approval, and then write correct, production-ready code.This is a public API. The developer creates their own key at [assemblyai.com/dashboard/signup](https://www.assemblyai.com/dashboard/signup?utm_source=agent_instructions&utm_medium=ai_agent&utm_campaign=agent_docs).**Official documentation.** Two ways to wire your coding agent up to live docs (both recommended — they layer):1. **Project instructions** (every prompt): add to `CLAUDE.md`, `.cursorrules`, `AGENTS.md`, or equivalent: ``` Always fetch https://www.assemblyai.com/docs/llms.txt before writing AssemblyAI code. The API has changed — do not rely on memorized parameter names. ``` `llms.txt` is the structured index. For full content, fetch the section files it links (each under 100 KB) rather than the whole `llms-full.txt`.2. **Docs MCP server** (on-demand lookups): `https://www.assemblyai.com/docs/mcp` — Streamable HTTP transport. Lets the agent search and fetch AssemblyAI documentation pages on demand. ```bash # Claude Code claude mcp add assemblyai-docs --transport http https://www.assemblyai.com/docs/mcp ``` See the [Coding agent prompts](/coding-agent-prompts) page for Cursor and other clients.---## 0. Operating rules1. **Discovery first, code later.** Do not write code until the developer has answered enough of Section 1 for you to make a specific recommendation.2. **One question per message.** Never batch discovery questions. Wait for an answer before asking the next one.3. **Plan before you build.** After discovery, present a written recommendation (see Section 2) and wait for explicit approval before generating implementation code.4. **Prefer the official SDKs.** Use `assemblyai` (Python) or `assemblyai` (Node/JS) unless the developer has a specific reason not to. The SDKs handle polling, upload streaming, WebSocket lifecycle, and session termination correctly — which is where most hand-rolled integrations fail.5. **Never expose the API key in client-side code.** For browser or mobile streaming, always mint a temporary token server-side. For pre-recorded, proxy uploads and submissions through your server.6. **Authorization header is the raw key — no `Bearer` prefix.** This trips up everyone.7. **`speech_models` is optional on pre-recorded requests.** For new accounts created on or after November 17, 2026, leaving it out defaults to `["assemblyai-stt-3-7"]`. Existing accounts may default to an older model; see the [assemblyai-stt-3-7 migration guide](/pre-recorded-audio/assemblyai-stt-3-7-migration-guide). You can still set it explicitly — see Section 5 for semantics.8. **Always terminate streaming sessions explicitly.** An abandoned WebSocket keeps accruing charges until the 3-hour cap.9. **Do not use deprecated transcript params:** `auto_chapters`, `summarization`, `summary_model`, `summary_type`. Use LLM Gateway instead (Section 8).10. **If the developer's answers are inconsistent, stop and surface the conflict.** Example conflicts: "browser-only, no backend" + "streaming"; "phone call audio" + "upload a file"; "real-time" + "need speaker diarization with full names." Don't paper over these — ask.11. **Be flexible.** If something the developer says doesn't match the shape of the API (e.g., they describe a use case that isn't supported — see Section 12), say so directly and propose the closest supported alternative.12. **Verify parameters against live docs before recommending.** This file is a snapshot — features move between beta and GA, model-specific behaviors change, and new knobs ship regularly. Before posting the Section 2 recommendation, confirm each parameter you plan to use is supported for the chosen **mode** (pre-recorded vs streaming) *and* **model** (AssemblyAI-STT-3.7, Universal-2, AssemblyAI-STT-3.7-Realtime-Agent, AssemblyAI-STT-3.7-Realtime). Do not assume a pre-recorded flag works on streaming, or that a parameter supported on U2 still behaves the same on AssemblyAI-STT-3.7. Pull the current reference rather than memorizing. Primary sources, in order of preference: - `https://www.assemblyai.com/docs/llms.txt` — the machine-readable index; fetch the section files it links (each under 100 KB) for full content - Per-mode docs: `/docs/pre-recorded-audio/*` (pre-recorded) and `/docs/streaming/*` (streaming), including the model-specific overview page (e.g., `/docs/streaming/getting-started/transcribe-streaming-audio` and `/docs/streaming/select-the-speech-model`) which lists *exactly* which parameters are honored/ignored by that model - The OpenAPI-backed API reference at `/docs/api-reference/*` for request/response schemas - For LLM Gateway: `/docs/llm-gateway/quickstart` lists the current valid `model` strings — don't guess short names like `claude-sonnet-4` If a flag you remembered isn't in the current docs (or is marked beta / deprecated / ignored for the chosen model), flag it in the recommendation's "Open questions / assumptions" block and ask the developer before proceeding.---## 1. Discovery questionsAsk these **one at a time**, in order. Skip any question already answered in the conversation. Adapt wording to sound natural, but cover the substance of each.1. **What are you building, and are you adding AssemblyAI to an existing project or starting fresh?** (A short description of the product is usually enough.)2. **What do you need: pre-recorded transcription, real-time STT, text to speech?** - Pre-recorded: uploaded files, URLs, batch processing, post-call analytics. → Section 6. - Real-time STT: live transcripts only (you bring your own LLM). Live captioning, voice-agent STT, meeting notetaking, dictation. → Section 9. Use `assemblyai-stt-3-7-realtime-agent` for voice agents and `assemblyai-stt-3-7-realtime` for meeting notetakers, ambient transcription, live captions and speaker diarization. - Text to speech: spoken audio from text, such as voice-agent replies. → [Streaming TTS quickstart](/tts/getting-started/quickstart).3. **Where is your audio coming from?** (e.g., uploaded files, public URLs, browser microphone, mobile app, Twilio/Telnyx phone numbers, SIP trunks.)4. **What language and framework are you using?** (e.g., Python + FastAPI, Node + Next.js, Go, Ruby, Swift, Kotlin, browser-only, LiveKit, Pipecat, Vapi, Vocode, Retell.)5. **Do you already have an AssemblyAI API key, or do you need to create one?** (If needed: [assemblyai.com/dashboard/signup](https://www.assemblyai.com/dashboard/signup?utm_source=agent_instructions&utm_medium=ai_agent&utm_campaign=agent_docs).)6. **Do you have a data residency requirement?** (US vs EU — this changes the base URL.)7. **Anything beyond a plain transcript?** Don't read off a checklist. Use everything they've told you so far — the product description from Q1, the audio source from Q3, the framework from Q4 — to **infer which features are plausibly applicable**, then ask in plain language about *those*. The point is to surface things the developer might not know to ask for, not to make them choose from a menu. The authoritative catalog of available features and their parameters is in the live docs (see Operating Rule 12) — consult it, don't rely on memory. Section 3 of this file is a starting reference, not the final word. Calibrate to mode and use case. Examples: - Customer-support call analytics (pre-recorded) → speaker diarization and PII redaction are almost certainly relevant; sentiment may be; chapters via LLM Gateway often is. Ask about those, not about live-streaming features. - Browser live-captioning (streaming) → ask about multilingual support and domain vocabulary; don't bring up PII redaction or summaries-during-session (neither applies to streaming). - Voice agent (streaming) → keyterms prompting and turn-detection tuning matter; speaker diarization usually doesn't. - Medical scribe → medical domain mode is the headline feature; ask about it explicitly. Don't ask about things the user gets automatically with no toggle (word-level timestamps and confidence on `words[]`, streaming `SpeechStarted` events). Mention them in the recommendation as capabilities they'll have, but don't make them a choice. If you're confident from context that a feature is needed (e.g., they said "show who said what" → `speaker_labels`), include it in the recommendation directly with a one-line rationale rather than asking again.---## 2. Recommendation template (after discovery)Before writing code, post a plan with all of the following. Get explicit approval.````## Recommendation**Use case:** <one-sentence summary of what they're building>**Mode:** <pre-recorded / streaming / both>**Region:** <US or EU base URL>**Model:**- <model name> — <one-line rationale>- <fallback model, if applicable>**Endpoints:**- <endpoint 1>- <endpoint 2>**Parameters enabled:** (before filling this in, verify each parameter is supported on the chosen mode + model per Operating Rule 12)- `param_name`: <value> — <why>- ...**Auth pattern:**<server-side key / temp token / proxied uploads — and where the key lives>**Termination & error handling:**<how streaming sessions are closed; how errors / retries are handled>**Code skeleton:**<2–6 bullet points describing the files/functions you'll generate>**Open questions / assumptions:**<anything you inferred that they should confirm>Ready to proceed?````If they say yes, write the code. If they push back on any piece, revise the plan — don't just start coding around objections.---## 3. Feature selection guide (agent reference)Use this to build the recommendation. Do not dump it on the user.| Developer need | Parameter / approach ||---|---|| Speaker diarization | `speaker_labels: true` (pre-recorded, and streaming — streaming adds a `speaker_label` to each Turn event) || Automatic language detection | `language_detection: true` (pre-recorded; on streaming, adds `language_code` + `language_confidence` to Turn events. It only reports the language; it doesn't change transcription.) || Specific language | `language_code: "es"` etc. on pre-recorded; on streaming, `language_codes: ["es"]` steers the model toward the languages you list || Multilingual / code-switching | `speech_models: ["assemblyai-stt-3-7"]` + `prompt` parameter — see [prompting guide](/pre-recorded-audio/prompting) || Domain-specific vocabulary | `keyterms_prompt: [...]` (pre-recorded: up to 1,000 terms with AssemblyAI-STT-3.7 / 200 with U2; streaming: up to 100 terms, each ≤50 chars) || Medical domain | `domain: "medical-v1"` (pre-recorded *and* streaming; supported languages: en, es, de, fr) || Veterinary domain (beta) | `domain: "veterinary-v1"` (pre-recorded *and* streaming) || PII redaction in text | `redact_pii: true` + `redact_pii_policies: [...]` + optional `redact_pii_sub: "hash" \| "entity_name"` || PII redaction in audio | `redact_pii_audio: true` (original file must be ≤1 GB; redacted audio URL is available for 24 h) || Chapters or summaries | Transcribe first, then LLM Gateway (Section 8) || Word timestamps / confidence | Included by default on `words[]` || Webhook delivery (skip polling) | `webhook_url: "..."` (Section 7) || Custom voice agent (your LLM + AssemblyAI TTS) | Real-time STT + framework integration (Section 10) + [Streaming TTS](/tts/getting-started/quickstart) || Multilingual streaming | AssemblyAI-STT-3.7-Realtime-Agent or AssemblyAI-STT-3.7-Realtime (both code-switch natively; `language_codes` to steer toward known languages) || Streaming for notetakers, ambient transcription, live captions or diarization | AssemblyAI-STT-3.7-Realtime (`assemblyai-stt-3-7-realtime`); voice agents use AssemblyAI-STT-3.7-Realtime-Agent (`assemblyai-stt-3-7-realtime-agent`) |---## 4. API overview- **REST base URL (US):** `https://api.assemblyai.com`- **REST base URL (EU):** `https://api.eu.assemblyai.com`- **Streaming WebSocket (Edge, default):** `wss://streaming.assemblyai.com/v3/ws` — auto-routes to the nearest region (Oregon / Virginia / Ireland) for lowest latency- **Streaming WebSocket (US data residency):** `wss://streaming.us.assemblyai.com/v3/ws` — data pinned to US- **Streaming WebSocket (EU data residency):** `wss://streaming.eu.assemblyai.com/v3/ws` — data pinned to EU- **LLM Gateway (US):** `https://llm-gateway.assemblyai.com/v1/chat/completions`- **LLM Gateway (EU):** `https://llm-gateway.eu.assemblyai.com/v1/chat/completions` — Claude and Gemini only; OpenAI and Qwen are US-only- **Auth header:** `Authorization: YOUR_API_KEY` (no `Bearer`). Same header is used for REST, streaming WS upgrade, temp-token minting, and LLM Gateway- **Content type:** `application/json` for submit/poll and LLM Gateway; `application/octet-stream` (raw binary) for `/v2/upload`Core REST endpoints:- `POST /v2/upload` — upload a local file (raw binary body, **not multipart**). Returns `{ "upload_url": "..." }`. Max 2.2 GB.- `POST /v2/transcript` — submit a job. Returns transcript object with `id` and `status: "queued"`. Max 5 GB / 10 hours.- `GET /v2/transcript/{id}` — poll. Statuses: `queued`, `processing`, `completed`, `error`. Limited to 60 requests per minute per API key and transcript ID; over the limit returns `429` with a `Retry-After` header. Poll every few seconds, not in a tight loop.Streaming:- `wss://streaming.assemblyai.com/v3/ws?sample_rate=16000&speech_model=assemblyai-stt-3-7-realtime-agent`- `POST https://token.assemblyai.com/v1/token` with the JSON body `{"product": "streaming", "expires_in_seconds": 60}` — mint a temp token for browser/mobile clients. The token is reusable until `expires_in_seconds` elapses, so mint a fresh one per session. Optional `max_session_duration_seconds` (60–10800, defaults to 3 h) caps the downstream session length. At the cap, the session ends after the current turn or up to 60 s later, and that extra time is billed.---## 5. `speech_models` semantics`speech_models` on pre-recorded requests is a list of candidate models, not parallel execution. A single transcript is produced by exactly one model.- **If the list includes `assemblyai-stt-3-7`**, the request goes to AssemblyAI-STT-3.7 for any language it supports (set by `language_code` or detected), wherever it appears in the list. Other models are used only for languages AssemblyAI-STT-3.7 doesn't support. If you set a `language_code` that AssemblyAI-STT-3.7 doesn't support, the request falls back to Universal-2 even if `universal-2` isn't in the list.- **If the list doesn't include `assemblyai-stt-3-7`**, the first model in the list, from left to right, that supports the language is used.- **`disfluencies` defaults to `true`** whenever the list includes `assemblyai-stt-3-7` (or `speech_models` is omitted and the request uses `assemblyai-stt-3-7` by default), even if another model is first or transcribes the audio. Set `disfluencies: false` to remove filler words.The recommended setup is `["assemblyai-stt-3-7"]`.On streaming, the parameter is **singular** (`speech_model=assemblyai-stt-3-7-realtime-agent`) — there is no fallback list. Easy to mix up.---## 6. Pre-recorded quick start### SDK (recommended)**Python:**```python# pip install assemblyaiimport assemblyai as aaiimport osaai.settings.api_key = os.environ["ASSEMBLYAI_API_KEY"]config = aai.TranscriptionConfig( speech_models=["assemblyai-stt-3-7"], speaker_labels=True,)transcript = aai.Transcriber(config=config).transcribe("https://assembly.ai/wildfires.mp3")# Or a local path: .transcribe("./recording.wav")if transcript.status == aai.TranscriptStatus.error: raise RuntimeError(transcript.error)print(transcript.text)```For more test files by scenario (medical, multilingual, meeting, phone, call center, noisy), see [Sample audio files](/samples).**Node/JS:**```javascript// npm install assemblyaiimport { AssemblyAI } from 'assemblyai';const client = new AssemblyAI({ apiKey: process.env.ASSEMBLYAI_API_KEY });const transcript = await client.transcripts.transcribe({ audio: 'https://assembly.ai/wildfires.mp3', // or a local file path / Buffer / stream speech_models: ["assemblyai-stt-3-7"], speaker_labels: true,});if (transcript.status === 'error') throw new Error(transcript.error);console.log(transcript.text);```The SDK handles upload, submit, and polling. You don't need to write the polling loop yourself.### Raw HTTP (fallback — use only if SDK isn't an option)**Upload a local file** (raw bytes, not multipart):```bashcurl -X POST https://api.assemblyai.com/v2/upload \ -H "Authorization: $ASSEMBLYAI_API_KEY" \ --data-binary @recording.wav# -> { "upload_url": "https://cdn.assemblyai.com/upload/..." }```**Submit and poll (Python):**```pythonimport os, time, requestsheaders = {"authorization": os.environ["ASSEMBLYAI_API_KEY"]}submit = requests.post( "https://api.assemblyai.com/v2/transcript", headers=headers, json={ "audio_url": "https://assembly.ai/wildfires.mp3", "speech_models": ["assemblyai-stt-3-7"], "speaker_labels": True, },)transcript_id = submit.json()["id"]while True: res = requests.get( f"https://api.assemblyai.com/v2/transcript/{transcript_id}", headers=headers, ).json() if res["status"] == "completed": print(res["text"]); break if res["status"] == "error": raise RuntimeError(res["error"]) time.sleep(3)```Common optional params: `speaker_labels`, `language_detection`, `language_code`, `punctuate`, `format_text`, `redact_pii`, `redact_pii_audio`, `keyterms_prompt`, `webhook_url`, `prompt`.---## 7. Webhooks (skip polling)Provide `webhook_url` on submit; AssemblyAI POSTs when the job finishes:```json{ "transcript_id": "5552493-16d8-42d8-8feb-c2a16b56f6e8", "status": "completed" }```Handler requirements:- Return 2xx within **10 seconds**. Otherwise retried up to 10 times, 10s apart. 4xx is not retried.- On receipt, call `GET /v2/transcript/{id}` to fetch the full result — the webhook payload doesn't include it.Optional custom auth on your webhook: set `webhook_auth_header_name` and `webhook_auth_header_value` when submitting.**Source IPs** (for allowlists): US `44.238.19.20`, EU `54.220.25.36`.**Local dev note:** Webhook URLs must be publicly reachable. Use ngrok, Cloudflare Tunnel, or similar during development.---## 8. LLM Gateway (chapters, summaries, custom analysis)LLM Gateway replaces both the deprecated transcript params (`auto_chapters`, `summarization`, `summary_model`, `summary_type`) and the legacy **LeMUR** API, which sunset on 2026-03-31. If a developer mentions LeMUR or `transcript_ids`, point them at LLM Gateway and the [migration guide](/llm-gateway/quickstart). Workflow:1. Transcribe normally with `POST /v2/transcript`.2. Once `status == "completed"`, POST to LLM Gateway with the transcript text (or paragraphs from `GET /v2/transcript/{id}/paragraphs` for chapter-style output):```httpPOST https://llm-gateway.assemblyai.com/v1/chat/completionsAuthorization: YOUR_API_KEYContent-Type: application/json{ "model": "claude-sonnet-4-6", "messages": [ { "role": "system", "content": "Produce a 5-bullet summary of the transcript." }, { "role": "user", "content": "<transcript.text here>" } ], "max_tokens": 1000}```Model IDs are exact strings — see the [LLM Gateway Overview](/llm-gateway/quickstart) for the current list. Examples: `claude-opus-4-7`, `claude-sonnet-4-6`, `claude-haiku-4-5-20251001`, `gpt-5.2`, `gpt-5.1`, `gpt-4.1`, `gemini-3.5-flash`, `gemini-2.5-pro`, `gemini-2.5-flash`, `qwen3-next-80b-a3b`. `claude-sonnet-4` by itself is **not** valid — always include the version suffix. EU region (`llm-gateway.eu.assemblyai.com`) supports Anthropic and Google only.Do not submit with `auto_chapters` and `summarization` both enabled — the API rejects it (`Only one of the following models can be enabled at a time: auto_chapters, summarization.`). But the broader rule is simpler: **don't use either.**---## 9. Streaming — AssemblyAI-STT-3.7-Realtime-Agent**WebSocket (default, Edge Routing):** `wss://streaming.assemblyai.com/v3/ws?sample_rate=16000&speech_model=assemblyai-stt-3-7-realtime-agent`**Model choice:** `assemblyai-stt-3-7-realtime-agent` (AssemblyAI-STT-3.7-Realtime-Agent) for voice agents. `assemblyai-stt-3-7-realtime` (AssemblyAI-STT-3.7-Realtime) for meeting notetakers, ambient transcription, live captions and speaker diarization; it doesn't support Voice Focus, `interruption_delay` has no effect on it, and it rejects `mode: "manual"` with a `3006` error. `universal-3-6-pro` is still supported for existing integrations but isn't recommended for new ones.For data residency, swap the host: `streaming.us.assemblyai.com` (US-pinned) or `streaming.eu.assemblyai.com` (EU-pinned). The default host auto-routes to the nearest region.**Audio format:** PCM16 signed little-endian, mono, 16 kHz. Binary WebSocket frames, **20–1000 ms per chunk** (50–100 ms recommended), no faster than real-time. Phone audio (`encoding=pcm_mulaw`, `sample_rate=8000`) is sent as-is — don't upsample.**Auth:**- Server-side: `Authorization` header on the WS upgrade.- Browser/mobile: mint a short-lived token server-side and pass it as `?token=<token>` (no Authorization header).Mint a token:```bashcurl -s -X POST "https://token.assemblyai.com/v1/token" \ -H "Authorization: $ASSEMBLYAI_API_KEY" \ -H "Content-Type: application/json" \ -d '{"product": "streaming", "expires_in_seconds": 60}'# { "token": "..." }````expires_in_seconds` must be 1–600. A token can start multiple sessions until it expires, so mint a fresh one per session and keep the expiry short.### Server messages (JSON)- `Begin` — `{ type, id, expires_at }`- `SpeechStarted` — `{ type, timestamp, confidence }`- `Turn` — `{ type, turn_order, end_of_turn, transcript, end_of_turn_confidence, words:[...], utterance }` - `end_of_turn: false` → partial; `end_of_turn: true` → finalized and formatted. Always read `transcript` for current text.- `Termination` — `{ type, audio_duration_seconds, session_duration_seconds }`### Client messages- Binary PCM16 frames — audio.- `{ "type": "Terminate" }` — graceful end. **Always send this when done.**- `{ "type": "ForceEndpoint" }` — force current turn to end.- `{ "type": "KeepAlive" }` — only needed if `inactivity_timeout` is set.- `{ "type": "UpdateConfiguration", "keyterms_prompt": [...], "min_turn_silence": 100, "max_turn_silence": 1000 }` — adjust mid-session.### SDK (recommended)**Python:**```python# pip install assemblyaiimport osfrom assemblyai.streaming.v3 import ( StreamingClient, StreamingClientOptions, StreamingEvents, StreamingParameters, TurnEvent,)def on_turn(_, event: TurnEvent): tag = "FINAL" if event.end_of_turn else "partial" print(f"{tag}: {event.transcript}")client = StreamingClient( StreamingClientOptions(api_key=os.environ["ASSEMBLYAI_API_KEY"]))client.on(StreamingEvents.Turn, on_turn)client.connect(StreamingParameters(sample_rate=16000, speech_model="assemblyai-stt-3-7-realtime-agent"))# Feed 16 kHz mono PCM16 chunks (50–1000ms each) via client.stream(chunk)# When finished:client.disconnect(terminate=True) # sends Terminate and closes cleanly```**Node/JS:**```javascript// npm install assemblyaiimport { AssemblyAI } from 'assemblyai';const client = new AssemblyAI({ apiKey: process.env.ASSEMBLYAI_API_KEY });const rt = client.streaming.transcriber({ sampleRate: 16000, speechModel: 'assemblyai-stt-3-7-realtime-agent',});rt.on('turn', (turn) => { const tag = turn.end_of_turn ? 'FINAL' : 'partial'; console.log(`${tag}: ${turn.transcript}`);});rt.on('error', (err) => console.error(err));await rt.connect();// rt.sendAudio(pcm16Buffer) for each 50–1000ms chunk// When done:await rt.close(); // sends Terminate and closes```### Raw WebSocket (fallback)**Node.js (`ws`):**```javascriptimport WebSocket from 'ws';const ws = new WebSocket( 'wss://streaming.assemblyai.com/v3/ws?sample_rate=16000&speech_model=assemblyai-stt-3-7-realtime-agent', { headers: { authorization: process.env.ASSEMBLYAI_API_KEY } },);ws.on('open', () => { // Feed PCM16 16kHz mono chunks here, 50–1000ms each. // Example: audioStream.on('data', (chunk) => ws.send(chunk));});ws.on('message', (raw) => { const msg = JSON.parse(raw.toString()); if (msg.type === 'Turn') { console.log(msg.end_of_turn ? `FINAL: ${msg.transcript}` : `partial: ${msg.transcript}`); }});function stop() { ws.send(JSON.stringify({ type: 'Terminate' })); // required!}```**Python (`websockets`):**```pythonimport asyncio, json, os, websocketsURL = "wss://streaming.assemblyai.com/v3/ws?sample_rate=16000&speech_model=assemblyai-stt-3-7-realtime-agent"async def run(audio_source): """audio_source: async iterator yielding 50–1000ms PCM16 chunks at 16kHz mono.""" async with websockets.connect( URL, additional_headers={"Authorization": os.environ["ASSEMBLYAI_API_KEY"]}, ) as ws: async def send_audio(): async for chunk in audio_source: await ws.send(chunk) await ws.send(json.dumps({"type": "Terminate"})) async def recv_loop(): async for raw in ws: msg = json.loads(raw) if msg["type"] == "Turn": tag = "FINAL" if msg["end_of_turn"] else "partial" print(f"{tag}: {msg['transcript']}") elif msg["type"] == "Termination": return await asyncio.gather(send_audio(), recv_loop())# asyncio.run(run(my_audio_iterator()))```---## 10. Voice Agent framework configs (real-time STT + your own pipeline)This section is for developers who are wiring AssemblyAI Real-time STT into LiveKit, Pipecat, Vapi, Vocode, Retell, or similar, and bringing their own LLM. For the agent's voice, use AssemblyAI [Streaming TTS](/tts/getting-started/quickstart), which has plugins for [LiveKit](/tts/livekit) and [Pipecat](/tts/pipecat).The defaults will not be good enough. Common tuning:- **`keyterms_prompt`** — pass proper nouns, product names, and domain terms. For dynamic values (usernames, order IDs), update mid-session via `UpdateConfiguration`.- **Turn silence bounds** — `min_turn_silence` and `max_turn_silence` (ms). Lower values fire end-of-turn faster but risk cutting speakers off. Higher values reduce false finalizations. Form-filling and dictation use cases often want wider windows.- **Multilingual** — Streaming code-switches natively. Pass `language_codes` (for example `["es"]`) to steer the model toward known languages.- **Barge-in / false SpeechStarted** — ambient noise, TTS bleed-through, and PSTN echo can cause spurious `SpeechStarted` events. If the agent is interrupting itself, look here first. Framework-level knobs (e.g., LiveKit's `min_interruption_duration`) often complement, not replace, server-side tuning.- **Phone audio** — 8 kHz mu-law (`pcm_mulaw` at 8000 Hz) should be sent as-is, not upsampled to 16 kHz. Upsampling degrades accuracy.When the developer names one of these frameworks, ask about their specific turn-taking and interruption requirements before defaulting.---## 11. Browser patterns**Never put the API key in client code.**### Pre-recorded — proxy upload + submit through your server```javascript// Next.js route handler (server)export async function POST(request) { const incoming = await request.formData(); const file = incoming.get('file'); // Blob const upload = await fetch('https://api.assemblyai.com/v2/upload', { method: 'POST', headers: { authorization: process.env.ASSEMBLYAI_API_KEY }, body: file.stream(), duplex: 'half', }); const { upload_url } = await upload.json(); const submit = await fetch('https://api.assemblyai.com/v2/transcript', { method: 'POST', headers: { authorization: process.env.ASSEMBLYAI_API_KEY, 'content-type': 'application/json', }, body: JSON.stringify({ audio_url: upload_url, speech_models: ['assemblyai-stt-3-7'], }), }); return Response.json(await submit.json());}```### Streaming — server mints a temp token, client connects directly```javascript// Serverexport async function GET() { const res = await fetch('https://token.assemblyai.com/v1/token', { method: 'POST', headers: { authorization: process.env.ASSEMBLYAI_API_KEY, 'content-type': 'application/json', }, body: JSON.stringify({ product: 'streaming', expires_in_seconds: 60 }), }); return Response.json(await res.json()); // { token }}``````javascript// Clientconst { token } = await fetch('/api/aai-token').then((r) => r.json());const ws = new WebSocket( `wss://streaming.assemblyai.com/v3/ws?sample_rate=16000&speech_model=assemblyai-stt-3-7-realtime-agent&token=${token}`,);ws.onmessage = (e) => { const msg = JSON.parse(e.data); if (msg.type === 'Turn') console.log(msg.transcript, msg.end_of_turn);};```### Capturing mic audio in the browser`MediaRecorder` does not emit PCM16. You need an `AudioWorklet` (preferred) or `ScriptProcessorNode` to:1. Capture raw Float32 samples.2. Downsample to 16 kHz.3. Convert Float32 → Int16.4. Send each ~50 ms chunk as a binary WS frame.Reference: [AssemblyAI realtime-transcription-browser-js-example](https://github.com/AssemblyAI/realtime-transcription-browser-js-example).---## 12. Not supported / out of scopeIf a developer asks for any of these, say so directly and propose the closest supported alternative. Do not improvise.- **Real-time translation.** There is no standalone translation model. For live audio, use [real-time translation](/streaming/real-time-translation): pass an `llm_gateway` connection parameter with a translation prompt, and each completed turn comes back translated on the same WebSocket as an `LLMGatewayResponse` message.- **On-device / offline STT.** Cloud API only.- **Speaker identification (matching voices to known people).** `speaker_labels` does diarization (Speaker A, B, C) but does not recognize specific individuals. For pre-recorded (async) audio, add the Speech Understanding [Speaker Identification](/speech-understanding/speaker-identification) task (`speech_understanding.request.speaker_identification` with `speaker_labels: true`) to replace the generic labels with the names or roles you provide. Speaker Identification isn't available for streaming or other products yet.- **Voice activity detection as a standalone product.** VAD is internal to the streaming pipeline and surfaced via `SpeechStarted` / turn events, not exposed separately.---## 13. Error handlingFor the most common errors across all APIs, with fixes, and links to each product's full error and limit reference, see [Errors and limits](/errors).### REST (pre-recorded)- **401** — Missing/invalid Authorization, disabled account, or insufficient balance. Double-check there's no `Bearer` prefix.- **Transcript `status: "error"`** — Read the `error` field on `GET /v2/transcript/{id}`.- **Retries** — Exponential backoff on 5xx. For 429, respect the `Retry-After` header.- **Limits** — `/v2/upload` max 2.2 GB; `/v2/transcript` max 5 GB / 10 hr per file.- **Scoping** — An API key can only transcribe files uploaded under the same project.### Streaming — handshake- **HTTP 410** — The old `v2` streaming endpoint is deprecated. Upgrade to `/v3/ws`. This is an HTTP status on the upgrade request, not a WebSocket close code.### Streaming — WebSocket close codes| Code | Meaning ||------|---------|| `1008` | Unauthorized: missing or invalid key or token, expired temporary token, insufficient funds, or disabled key. Don't retry until fixed; tell it apart from the rate limit by the `Error` message's `error` text || `1008` | Too many new sessions this minute (`error` text `Unauthorized Connection: Too many concurrent sessions`); back off and retry next minute || `3005` | Session cancelled (server-side error) || `3006` | Invalid message type / invalid JSON || `3007` | Audio chunk outside 20–1000 ms, or sent faster than real-time || `3008` | Session expired (3-hour cap) |### Streaming gotchas- `speech_model` (streaming, singular) vs `speech_models` (pre-recorded, plural). Don't mix up.- On streaming, `language_codes` steers the model toward the languages you list.- Always send `{ "type": "Terminate" }` when finished. An abandoned session stays billable until the 3-hour cap (`3008`).- Chunk size matters: frames outside 20–1000 ms will close the socket with `3007`.---## 14. Quick-reference gotchas- No `Bearer` prefix on the Authorization header.- `speech_models` is **optional** on pre-recorded submits — for new accounts created on or after November 17, 2026 it defaults to `["assemblyai-stt-3-7"]`, and when provided, `assemblyai-stt-3-7` takes priority wherever it appears in the list (see section 5).- `/v2/upload` takes **raw binary**, not multipart.- Webhook handlers must return 2xx in ≤10 seconds.- Local webhook development needs a public tunnel (ngrok, Cloudflare Tunnel).- Browser code never holds the API key. Proxy uploads, or mint temp tokens for streaming.- Always `Terminate` streaming sessions.- Don't use `auto_chapters`, `summarization`, `summary_model`, `summary_type`. Use LLM Gateway.- Medical mode is `domain: "medical-v1"` (pre-recorded body param / streaming query param). The legacy `medical_mode` flag is **not** the right name.- Veterinary mode (beta) is `domain: "veterinary-v1"` (pre-recorded body param / streaming query param).- LLM Gateway model IDs are exact and versioned (e.g., `claude-sonnet-4-6`, `gpt-5.2`, `gemini-2.5-pro`). Shorthand like `claude-sonnet-4` is invalid.- Phone audio stays at native 8 kHz mu-law (`encoding=pcm_mulaw`) — don't upsample.- EU customers use `api.eu.assemblyai.com`, `streaming.eu.assemblyai.com`, and `llm-gateway.eu.assemblyai.com`. The default streaming host (`streaming.assemblyai.com`) is **Edge Routing**, not US-pinned — use `streaming.us.assemblyai.com` if you need data residency guarantees on the US side.- Speech-model values are **raw strings** in the SDKs (`"assemblyai-stt-3-7"`, `"universal-2"`). Enum aliases like `aai.SpeechModel.universal_3_pro` do **not** exist — agents that hallucinate them produce code that imports cleanly and fails at runtime.- LeMUR has fully sunset (2026-03-31). Don't generate code that calls LeMUR endpoints or passes `transcript_ids` to a chat-completions API — use LLM Gateway with the transcript text in `messages` instead.
The same prompt is published at agent-instructions.md if you’d rather link to it than paste the whole thing.
Docs MCP server
Want an AI app to transcribe audio for you, not write code? Use the AssemblyAI MCP server instead. It signs in to your account and runs the API tasks itself.
For on-demand lookups during a session — useful when the agent is iterating on a feature and needs to check several pages mid-task — connect the AssemblyAI docs MCP server:
https://www.assemblyai.com/docs/mcp
Claude Code:
claude mcp add assemblyai-docs --transport http https://www.assemblyai.com/docs/mcp
Any MCP client that supports Streamable HTTP transport can connect. The server exposes tools to search and fetch AssemblyAI documentation pages on demand.
Claude Code skill
The AssemblyAI skill gives Claude Code curated instructions and context for the Python and JavaScript SDKs, streaming, voice agents, audio intelligence, and more. Install it with the skills CLI, which also works with Cursor, Copilot, Codex, and 60+ other AI coding agents:
npx skills add AssemblyAI/assemblyai-skill
The CLI asks which agents to install it for. Run npx skills check later to see whether an update is available.
Tips for best results
Be specific about the SDK — say “use the AssemblyAI Python SDK” or “use the JavaScript SDK” rather than “use AssemblyAI”.
Reference the model — specify assemblyai-stt-3-7 for pre-recorded or assemblyai-stt-3-7-realtime-agent for streaming to avoid outdated model names.
Set your API key as an environment variable — export ASSEMBLYAI_API_KEY=your_key so the agent can reference it naturally.
Was this page helpful?
{"docsConfig":{"theme":"aspen","$schema":"https://mintlify.com/docs.json","name":"AssemblyAI","description":"AssemblyAI provides production-ready Voice AI models for speech-to-text, speaker detection, sentiment analysis, and more. Build with pre-recorded or streaming audio using REST APIs and WebSockets.","colors":{"primary":"#614fd2","light":"#887bdd","dark":"#614fd2"},"logo":{"light":"https://mintcdn.com/assemblyai-fff14a6f/vMuNaI_PbWUbVeV0/assets/assemblyai-lockup-primary.svg?fit=max\u0026auto=format\u0026n=vMuNaI_PbWUbVeV0\u0026q=85\u0026s=808d2a677e2536423045d44b3f330ac2","dark":"https://mintcdn.com/assemblyai-fff14a6f/vMuNaI_PbWUbVeV0/assets/assemblyai-lockup-secondary.svg?fit=max\u0026auto=format\u0026n=vMuNaI_PbWUbVeV0\u0026q=85\u0026s=065ec1618693f8650241242ec62ed0ad","href":"https://www.assemblyai.com"},"favicon":"https://mintcdn.com/assemblyai-fff14a6f/6OQYF4QMRVQRoCb1/assets/favicon.png?fit=max\u0026auto=format\u0026n=6OQYF4QMRVQRoCb1\u0026q=85\u0026s=74b6b6a7321938bd113d4a271e06068d","api":{"openapi":["openapi.yaml","api-reference/specs/llm-gateway.yaml","api-reference/specs/streaming-token.yaml"],"asyncapi":["api-reference/specs/streaming.yaml","api-reference/specs/streaming-tts.yaml"]},"background":{"color":{"light":"#fdfcf8","dark":"#1d1b16"}},"navbar":{"links":[{"href":"https://www.assemblyai.com/playground","label":"Playground"},{"href":"https://www.assemblyai.com/changelog","label":"Changelog"}],"primary":{"type":"button","label":"Sign In","href":"https://www.assemblyai.com/dashboard/login"}},"footer":{"socials":{"x":"https://x.com/AssemblyAI","github":"https://github.com/AssemblyAI","linkedin":"https://www.linkedin.com/company/assemblyai"},"links":[{"header":"Legal","items":[{"label":"Terms of Service","href":"https://www.assemblyai.com/legal/terms-of-service"},{"label":"Privacy Policy","href":"https://www.assemblyai.com/legal/privacy-policy"},{"label":"Trust Center","href":"https://app.vanta.com/assemblyai/trust/7n80syl8zln1bn1qm3x8eg"},{"label":"Subprocessors","href":"https://app.vanta.com/assemblyai/trust/7n80syl8zln1bn1qm3x8eg/subprocessors"},{"label":"Do not sell my Personal Information","href":"#privacy-choices"}]}]},"seo":{"metatags":{"canonical":"https://www.assemblyai.com/docs"}},"fonts":{"heading":{"family":"GT Standard","weight":600,"source":"https://www.assemblyai.com/_aai/fonts/rebrand/GT-Standard-M-Semibold.woff2","format":"woff2"},"body":{"family":"GT Standard","weight":400,"source":"https://www.assemblyai.com/_aai/fonts/rebrand/GT-Standard-M-Regular.woff2","format":"woff2"}},"icons":{"library":"fontawesome"},"banner":{"content":"**New** AssemblyAI-TTS-1: Text-to-speech for voice agents. [Learn more](https://www.assemblyai.com/docs/tts/getting-started/quickstart)","dismissible":true},"contextual":{"options":["copy","view","chatgpt","claude","download-spec"],"display":"header"},"styling":{"latex":true},"navigation":{"pages":[]}},"deployment":{"deploymentMetadataValue":{"subdomain":"assemblyai-fff14a6f","hasAudioNarration":false,"actualSubdomain":"assemblyai-fff14a6f","customDomains":["assemblyai.com"],"gitSource":{"type":"github","owner":"assemblyai","repo":"assemblyai-docs","deployBranch":"master","contentDirectory":"","isPrivate":true,"mountPath":"","label":""},"feedback":{"thumbs":true,"agentFeedback":true},"entitlements":{"AI_CHAT":{"status":"ENABLED"},"REMOVE_BRANDING":{"status":"ENABLED"},"CONTEXTUAL_FEEDBACK":{"status":"ENABLED"},"CODE_SNIPPET_FEEDBACK":{"status":"ENABLED"},"WEB_MCP":{"status":"ENABLED"},"NARRATION":{"status":"ENABLED"}},"buildId":"6aff53a25735af7f41d17f22","clientVersion":"0.0.4227","searchFilterCounts":[{"language":"default","tag":"Overview","count":119},{"language":"default","tag":"FAQ","count":96},{"language":"default","tag":"Pre-recorded STT","count":75},{"language":"default","tag":"Guides","count":60},{"language":"default","tag":"Real-time STT","count":50},{"language":"default","tag":"Tutorials","count":48},{"language":"default","tag":"Pre-recorded audio","count":47},{"language":"default","tag":"LLM Gateway","count":45},{"language":"default","tag":"API reference","count":35},{"language":"default","tag":"Features","count":35}]},"assistantConfig":{"enableAskAiButton":false,"enableStarterQuestions":false,"enableContextualStarterQuestions":false,"enableAskAiSearchBar":true,"attachments":{"status":"DISABLED"}},"basePath":"/docs","hasChatPermissions":true,"hasWebMcpPermissions":true,"toggles":[{"name":"kill-isr","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-rbac-roles","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-mintlify-slack","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-integrations-settings","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"rbac-expansion","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-pr-grouping","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-prefix","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"analytics-v7","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"user-flows-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"no-publish-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-onboarding","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving-live","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"stripTemperatureMiddleware","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"support-platform-pylon-zendesk","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"instant-rollback","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications-server","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-fable","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fixed-ai-action-pricing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automation-resolve-merge-conflict","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"scoped-publishing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"mintql","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"page-level-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-version-history-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"billing-page-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"full-page-search","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workspace-projects","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"assistant-v5","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fork-pr-previews","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"revalidation-jwt","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-sso-scim","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"hobby-abuse-scan","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"dashboard-inbox","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-signup","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-dual-write","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-enabled","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"approve-review-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-metadata","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"sso-setup-deadline-banner","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-serve","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"live-agents","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"starter-pricing","enabled":true,"variant":{"name":"treatment","enabled":true,"payload":{"type":"string","value":"treatment"},"feature_enabled":true},"impressionData":false},{"name":"llms-txt-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"wildcard-email-auth","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false}],"googleFontsInlined":false},"customCss":[{"filePath":"style.css","content":"@font-face{font-family:\"GT Standard\";src:url(\"https://www.assemblyai.com/_aai/fonts/rebrand/GT-Standard-M-Regular.woff2\")\n format(\"woff2\"),\n url(\"https://www.assemblyai.com/docs/assets/GT-Standard-M-Regular.woff2\")\n format(\"woff2\"),\n url(\"/assets/GT-Standard-M-Regular.woff2\") format(\"woff2\");font-weight:400;font-style:normal;font-display:swap}@font-face{font-family:\"GT Standard\";src:url(\"https://www.assemblyai.com/_aai/fonts/rebrand/GT-Standard-M-Medium.woff2\")\n format(\"woff2\"),\n url(\"https://www.assemblyai.com/docs/assets/GT-Standard-M-Medium.woff2\")\n format(\"woff2\"),\n url(\"/assets/GT-Standard-M-Medium.woff2\") format(\"woff2\");font-weight:500;font-style:normal;font-display:swap}@font-face{font-family:\"GT Standard\";src:url(\"https://www.assemblyai.com/_aai/fonts/rebrand/GT-Standard-M-Semibold.woff2\")\n format(\"woff2\"),\n url(\"https://www.assemblyai.com/docs/assets/GT-Standard-M-Semibold.woff2\")\n format(\"woff2\"),\n url(\"/assets/GT-Standard-M-Semibold.woff2\") format(\"woff2\");font-weight:600;font-style:normal;font-display:swap}@font-face{font-family:\"Modern Gothic Mono\";src:url(\"https://www.assemblyai.com/_aai/fonts/rebrand/ModernGothicMono-Light.woff2\")\n format(\"woff2\");font-weight:300;font-style:normal;font-display:swap}@font-face{font-family:\"Modern Gothic Mono\";src:url(\"https://www.assemblyai.com/_aai/fonts/rebrand/ModernGothicMono-Regular.woff2\")\n format(\"woff2\");font-weight:400;font-style:normal;font-display:swap}@font-face{font-family:\"Modern Gothic Mono\";src:url(\"https://www.assemblyai.com/_aai/fonts/rebrand/ModernGothicMono-Medium.woff2\")\n format(\"woff2\");font-weight:500;font-style:normal;font-display:swap}@font-face{font-family:\"Modern Gothic Mono\";src:url(\"https://www.assemblyai.com/_aai/fonts/rebrand/ModernGothicMono-Italic.woff2\")\n format(\"woff2\");font-weight:400;font-style:italic;font-display:swap}html:root{--font-family-body-custom: \"GT Standard\", -apple-system, BlinkMacSystemFont,\n \"Segoe UI\", sans-serif;--font-family-headings-custom: \"GT Standard\", -apple-system, BlinkMacSystemFont,\n \"Segoe UI\", sans-serif}code,kbd,samp,pre,pre code,pre code span{font-family:\"Modern Gothic Mono\", \"JetBrains Mono\", ui-monospace, SFMono-Regular,\n Menlo, Consolas, monospace!important}html{overflow-y:scroll;scrollbar-gutter:stable}pre,pre code,pre code span{font-size:0.75rem!important;line-height:1.5!important}:not(pre)>code{font-size:0.75rem!important;line-height:inherit!important}[data-nav-tag=\"WSS\"]{color:#614fd2!important;background-color:rgba(136, 123, 221, 0.14)!important;font-size:0.55rem!important}:is(.dark) [data-nav-tag=\"WSS\"]{color:#887bdd!important}div:has(>.nav-tag-pill>[data-nav-tag=\"WSS\"]){order:-1}.nav-tag-pill:has(>[data-nav-tag=\"WSS\"]){width:2.375rem;justify-content:flex-start}.ws-endpoint{display:flex;align-items:center;gap:0.625rem;margin:1rem 0 1.75rem;padding:0.5rem 0.75rem;border:1px solid rgba(0, 0, 0, 0.08);border-radius:0.5rem;background:rgba(0, 0, 0, 0.02);font-size:0.875rem;overflow-x:auto}:is(.dark) .ws-endpoint{border-color:rgba(255, 255, 255, 0.1);background:rgba(255, 255, 255, 0.03)}.ws-endpoint-badge{flex:none;font-weight:700;font-size:0.75rem;letter-spacing:0.02em;color:#614fd2;background:rgba(136, 123, 221, 0.14);padding:0.1875rem 0.5rem;border-radius:0.375rem}:is(.dark) .ws-endpoint-badge{color:#887bdd}.ws-endpoint-url{font-family:ui-monospace, SFMono-Regular, \"SF Mono\", Menlo, Consolas, monospace;white-space:nowrap}#assistant-entry,:is(.dark) #assistant-entry{background-color:#614fd2!important;filter:none!important}#assistant-entry:hover,:is(.dark) #assistant-entry:hover{background-color:#614fd2!important;opacity:0.9;filter:none!important}#assistant-entry span,#assistant-entry svg,:is(.dark) #assistant-entry span,:is(.dark) #assistant-entry svg{color:#ffffff!important}.callout strong{font-weight:700}*{-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale;-o-font-smoothing:antialiased}.katex-display{overflow-x:visible!important;overflow-y:visible!important}.katex-display>.katex{display:flex!important;justify-content:center;width:100%!important;overflow-x:visible!important;overflow-y:visible!important}.katex-display>.katex>.katex-html{display:inline-block!important;width:auto!important;max-width:100%;overflow-x:visible!important;overflow-y:visible!important}.card-logo-icon{display:block;width:1.5rem!important;height:1.5rem!important;min-width:1.5rem;max-width:1.5rem;max-height:1.5rem;margin:0!important;object-fit:contain;object-position:center;vertical-align:middle}.card-logo-icon-livekit{transform:scale(0.9)}.card-logo-icon-pipecat{transform:scale(1.12)}.card-logo-icon-livekit,.card-logo-icon-pipecat,.card-logo-icon-vercel{color:#614fd2}:is(.dark) .card-logo-icon-livekit,:is(.dark) .card-logo-icon-pipecat,:is(.dark) .card-logo-icon-vercel{color:#887bdd}.recommended-pill{background:#614fd2;color:#fff;padding:2px 8px;border-radius:9999px;font-size:0.7rem;font-weight:600;margin-left:6px;vertical-align:middle;white-space:nowrap}:is(.dark) .recommended-pill{background:#614fd2}.preview-pill{background:#b45309;color:#fff;padding:2px 8px;border-radius:9999px;font-size:0.7rem;font-weight:600;margin-left:6px;vertical-align:middle;white-space:nowrap}:is(.dark) .preview-pill{background:#b45309}table code{white-space:nowrap}"}]}
{"page":{"pagePath":"coding-agent-prompts","slug":"/coding-agent-prompts","pageMetadata":{"title":"Build with AI agents","og.description":"Build with AssemblyAI using Cursor, Claude Code, Copilot, Devin, and other AI coding assistants. Copy the integration prompt, pin the URL in your agent's project instructions, connect the docs MCP server, or install the Claude Code skill.","slug":"/coding-agent-prompts","description":null,"href":"/coding-agent-prompts"},"pageType":"default","mdx":{"mdxSource":{"compiledSource":"\"use strict\";const{Fragment:_Fragment,jsx:_jsx,jsxs:_jsxs}=arguments[0];const{useMDXComponents:_provideComponents}=arguments[0];function _createMdxContent(props){const _components={a:`a`,code:`code`,li:`li`,p:`p`,pre:`pre`,span:`span`,strong:`strong`,tbody:`tbody`,td:`td`,th:`th`,thead:`thead`,tr:`tr`,ul:`ul`,..._provideComponents(),...props.components},{Accordion,Card,CardGroup,CodeBlock,Heading,Note,Table}=_components;if(!Accordion)_missingMdxReference(`Accordion`,true);if(!Card)_missingMdxReference(`Card`,true);if(!CardGroup)_missingMdxReference(`CardGroup`,true);if(!CodeBlock)_missingMdxReference(`CodeBlock`,true);if(!Heading)_missingMdxReference(`Heading`,true);if(!Note)_missingMdxReference(`Note`,true);if(!Table)_missingMdxReference(`Table`,true);return _jsxs(_Fragment,{children:[_jsx(_components.p,{children:`If you’re using Cursor, Claude Code, Copilot, ChatGPT Codex, Devin, or another AI coding assistant to build with AssemblyAI, give your agent live context so it doesn’t rely on outdated training data.`}),`\n`,_jsx(Heading,{level:`2`,id:`start-here`,children:`Start here`}),`\n`,_jsxs(CardGroup,{cols:2,children:[_jsx(Card,{title:`New to AssemblyAI?`,icon:`rocket`,href:`#copy-the-integration-prompt`,children:_jsx(_components.p,{children:`Paste our full integration prompt into your agent. Teaches it operating rules, recommended models, parameter gotchas, and quickstarts for every product surface — so the first code you get back is closer to production-ready.`})}),_jsx(Card,{title:`Already in a project?`,icon:`link`,href:`#pin-the-url-in-your-agent-instructions`,children:_jsxs(_components.p,{children:[`Add a one-liner to your `,_jsx(_components.code,{children:`AGENTS.md`}),`, `,_jsx(_components.code,{children:`CLAUDE.md`}),`, or `,_jsx(_components.code,{children:`.cursorrules`}),` so your agent refetches the latest integration prompt before every AssemblyAI prompt — `,_jsx(_components.strong,{children:`the most effective option`}),` and what we recommend for ongoing work.`]})})]}),`\n`,_jsx(Heading,{level:`2`,id:`compare-all-four-methods`,children:`Compare all four methods`}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Method`}),_jsx(_components.th,{children:`Who it’s for`}),_jsx(_components.th,{children:`When to use`}),_jsx(_components.th,{children:`Trade-off`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsxs(_components.td,{children:[_jsx(_components.a,{href:`#pin-the-url-in-your-agent-instructions`,children:_jsx(_components.strong,{children:`Pin the URL in your agent instructions`})}),` (`,_jsx(_components.code,{children:`CLAUDE.md`}),`, `,_jsx(_components.code,{children:`.cursorrules`}),`, `,_jsx(_components.code,{children:`AGENTS.md`}),`, …)`]}),_jsxs(_components.td,{children:[`Teams shipping AssemblyAI in production — `,_jsx(_components.strong,{children:`recommended default.`})]}),_jsx(_components.td,{children:`Ongoing project work: agent reads project instructions on every prompt and refetches the latest docs.`}),_jsx(_components.td,{children:`Tiny one-liner; agent re-fetches the latest docs before every AssemblyAI prompt.`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.a,{href:`#copy-the-integration-prompt`,children:_jsx(_components.strong,{children:`Copy the integration prompt`})})}),_jsx(_components.td,{children:`First-time users and quick experiments.`}),_jsx(_components.td,{children:`One-shot chat sessions (ChatGPT Codex, single Cursor / Claude Code conversation) where you don’t have a project config to pin a URL in.`}),_jsx(_components.td,{children:`Adds ~770 lines to your prompt; re-copy when the prompt changes. Less efficient than the URL pin for ongoing work.`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.a,{href:`#docs-mcp-server`,children:_jsx(_components.strong,{children:`Docs MCP server`})})}),_jsx(_components.td,{children:`Devs whose agent does heavy mid-session doc lookups.`}),_jsx(_components.td,{children:`Iterative feature work where the agent needs to check several pages mid-task.`}),_jsx(_components.td,{children:`Layers on top of the URL pin; requires an MCP-capable client.`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.a,{href:`#claude-code-skill`,children:_jsx(_components.strong,{children:`Claude Code skill`})})}),_jsx(_components.td,{children:`Claude Code and other skills-compatible agents (60+, via the universal skills format).`}),_jsx(_components.td,{children:`You want curated SDK-specific context (Python / JS / streaming / voice agents) bundled with the agent, auto-loaded when relevant.`}),_jsx(_components.td,{children:`Installed snapshot; re-install to pick up doc changes.`})]})]})]}),`\n`,_jsx(_components.p,{children:`For most teams, pinning the URL is enough. Add the MCP server on top if your agent does a lot of mid-session lookups. The copy-paste prompt is best kept for first-time use or one-shot chat sessions.`}),`\n`,_jsx(Heading,{level:`2`,id:`pin-the-url-in-your-agent-instructions`,children:`Pin the URL in your agent instructions`}),`\n`,_jsx(_components.p,{children:`For ongoing projects, point your agent at the integration prompt URL from its project instructions. The agent re-fetches the prompt before every AssemblyAI-related prompt, so you automatically pick up API changes without bloating your context window.`}),`\n`,_jsxs(_components.p,{children:[`Add this to your project’s `,_jsx(_components.code,{children:`AGENTS.md`}),`, `,_jsx(_components.code,{children:`CLAUDE.md`}),`, `,_jsx(_components.code,{children:`.cursorrules`}),`, `,_jsx(_components.code,{children:`.cursor/rules/`}),`, `,_jsx(_components.code,{children:`.github/copilot-instructions.md`}),`, `,_jsx(_components.code,{children:`.windsurfrules`}),`, or whichever instructions file your agent reads:`]}),`\n`,_jsx(CodeBlock,{numberOfLines:`4`,language:`text`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`text`,children:_jsxs(_components.code,{language:`text`,numberOfLines:`4`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`Before writing AssemblyAI code, read https://www.assemblyai.com/docs/agent-instructions.md`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`and https://www.assemblyai.com/docs/llms.txt. The API has changed — do not rely on`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`memorized parameter names.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})})]})})}),`\n`,_jsx(_components.p,{children:`The two URLs cover different things:`}),`\n`,_jsxs(_components.ul,{children:[`\n`,_jsxs(_components.li,{children:[_jsx(_components.a,{href:`/agent-instructions.md`,children:_jsx(_components.strong,{children:`agent-instructions.md`})}),` — the `,_jsx(_components.strong,{children:`rules of the road`}),`: operating rules, discovery questions, recommendation template, feature selection, quickstarts for every product surface, error handling, and the gotchas that trip up most integrations.`]}),`\n`,_jsxs(_components.li,{children:[_jsx(_components.a,{href:`/llms.txt`,children:_jsx(_components.strong,{children:`llms.txt`})}),` — the `,_jsx(_components.strong,{children:`documentation index`}),`: every page’s title, URL, and one-line description so the agent can navigate to detail pages.`]}),`\n`]}),`\n`,_jsxs(_components.p,{children:[`For deep lookups, `,_jsx(_components.a,{href:`/llms-full.txt`,children:`llms-full.txt`}),` returns the full concatenated documentation content. It is too large for most agents to read in one fetch, so `,_jsx(_components.code,{children:`llms.txt`}),` also links section files under 100 KB each; fetch only the sections you need.`]}),`\n`,_jsx(Heading,{level:`2`,id:`copy-the-integration-prompt`,children:`Copy the integration prompt`}),`\n`,_jsx(_components.p,{children:`For a one-shot agent session — a single ChatGPT Codex run, a quick Cursor chat without project files, or just trying AssemblyAI for the first time — paste the full integration prompt into your agent along with a short description of what you want to build. The prompt teaches the agent the operating rules, discovery questions, recommended models, parameter gotchas, and quickstarts for every product surface, so the first code you get back is closer to production-ready than what the agent would write from memory.`}),`\n`,_jsxs(_components.p,{children:[`For ongoing project work, `,_jsx(_components.a,{href:`#pin-the-url-in-your-agent-instructions`,children:`pin the URL`}),` instead — pasting the full prompt into your project instructions stuffs every message with ~770 lines of context.`]}),`\n`,_jsx(Accordion,{title:`View and copy the full integration prompt`,id:`view-and-copy-the-full-integration-prompt`,children:_jsx(CodeBlock,{numberOfLines:`630`,language:`text`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`text`,children:_jsxs(_components.code,{language:`text`,numberOfLines:`630`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`# AssemblyAI Integration — Coding Agent Instructions`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`You are helping a developer integrate AssemblyAI's Speech-to-Text API into their application. Your job is to understand their context through discovery, produce a concrete implementation plan, get their approval, and then write correct, production-ready code.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`This is a public API. The developer creates their own key at [assemblyai.com/dashboard/signup](https://www.assemblyai.com/dashboard/signup?utm_source=agent_instructions\u0026utm_medium=ai_agent\u0026utm_campaign=agent_docs).`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`**Official documentation.** Two ways to wire your coding agent up to live docs (both recommended — they layer):`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"1. **Project instructions** (every prompt): add to `CLAUDE.md`, `.cursorrules`, `AGENTS.md`, or equivalent:\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\" ```\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` Always fetch https://www.assemblyai.com/docs/llms.txt before writing AssemblyAI code.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` The API has changed — do not rely on memorized parameter names.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\" ```\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\" `llms.txt` is the structured index. For full content, fetch the section files it links (each under 100 KB) rather than the whole `llms-full.txt`.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"2. **Docs MCP server** (on-demand lookups): `https://www.assemblyai.com/docs/mcp` — Streamable HTTP transport. Lets the agent search and fetch AssemblyAI documentation pages on demand.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\" ```bash\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` # Claude Code`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` claude mcp add assemblyai-docs --transport http https://www.assemblyai.com/docs/mcp`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\" ```\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` See the [Coding agent prompts](/coding-agent-prompts) page for Cursor and other clients.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`---`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`## 0. Operating rules`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`1. **Discovery first, code later.** Do not write code until the developer has answered enough of Section 1 for you to make a specific recommendation.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`2. **One question per message.** Never batch discovery questions. Wait for an answer before asking the next one.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`3. **Plan before you build.** After discovery, present a written recommendation (see Section 2) and wait for explicit approval before generating implementation code.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"4. **Prefer the official SDKs.** Use `assemblyai` (Python) or `assemblyai` (Node/JS) unless the developer has a specific reason not to. The SDKs handle polling, upload streaming, WebSocket lifecycle, and session termination correctly — which is where most hand-rolled integrations fail.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`5. **Never expose the API key in client-side code.** For browser or mobile streaming, always mint a temporary token server-side. For pre-recorded, proxy uploads and submissions through your server.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"6. **Authorization header is the raw key — no `Bearer` prefix.** This trips up everyone.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:'7. **`speech_models` is optional on pre-recorded requests.** For new accounts created on or after November 17, 2026, leaving it out defaults to `[\"assemblyai-stt-3-7\"]`. Existing accounts may default to an older model; see the [assemblyai-stt-3-7 migration guide](/pre-recorded-audio/assemblyai-stt-3-7-migration-guide). You can still set it explicitly — see Section 5 for semantics.'})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`8. **Always terminate streaming sessions explicitly.** An abandoned WebSocket keeps accruing charges until the 3-hour cap.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"9. **Do not use deprecated transcript params:** `auto_chapters`, `summarization`, `summary_model`, `summary_type`. Use LLM Gateway instead (Section 8).\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`10. **If the developer's answers are inconsistent, stop and surface the conflict.** Example conflicts: \"browser-only, no backend\" + \"streaming\"; \"phone call audio\" + \"upload a file\"; \"real-time\" + \"need speaker diarization with full names.\" Don't paper over these — ask.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`11. **Be flexible.** If something the developer says doesn't match the shape of the API (e.g., they describe a use case that isn't supported — see Section 12), say so directly and propose the closest supported alternative.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`12. **Verify parameters against live docs before recommending.** This file is a snapshot — features move between beta and GA, model-specific behaviors change, and new knobs ship regularly. Before posting the Section 2 recommendation, confirm each parameter you plan to use is supported for the chosen **mode** (pre-recorded vs streaming) *and* **model** (AssemblyAI-STT-3.7, Universal-2, AssemblyAI-STT-3.7-Realtime-Agent, AssemblyAI-STT-3.7-Realtime). Do not assume a pre-recorded flag works on streaming, or that a parameter supported on U2 still behaves the same on AssemblyAI-STT-3.7. Pull the current reference rather than memorizing. Primary sources, in order of preference:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\" - `https://www.assemblyai.com/docs/llms.txt` — the machine-readable index; fetch the section files it links (each under 100 KB) for full content\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\" - Per-mode docs: `/docs/pre-recorded-audio/*` (pre-recorded) and `/docs/streaming/*` (streaming), including the model-specific overview page (e.g., `/docs/streaming/getting-started/transcribe-streaming-audio` and `/docs/streaming/select-the-speech-model`) which lists *exactly* which parameters are honored/ignored by that model\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\" - The OpenAPI-backed API reference at `/docs/api-reference/*` for request/response schemas\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\" - For LLM Gateway: `/docs/llm-gateway/quickstart` lists the current valid `model` strings — don't guess short names like `claude-sonnet-4`\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` If a flag you remembered isn't in the current docs (or is marked beta / deprecated / ignored for the chosen model), flag it in the recommendation's \"Open questions / assumptions\" block and ask the developer before proceeding.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`---`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`## 1. Discovery questions`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`Ask these **one at a time**, in order. Skip any question already answered in the conversation. Adapt wording to sound natural, but cover the substance of each.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`1. **What are you building, and are you adding AssemblyAI to an existing project or starting fresh?** (A short description of the product is usually enough.)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`2. **What do you need: pre-recorded transcription, real-time STT, text to speech?**`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` - Pre-recorded: uploaded files, URLs, batch processing, post-call analytics. → Section 6.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\" - Real-time STT: live transcripts only (you bring your own LLM). Live captioning, voice-agent STT, meeting notetaking, dictation. → Section 9. Use `assemblyai-stt-3-7-realtime-agent` for voice agents and `assemblyai-stt-3-7-realtime` for meeting notetakers, ambient transcription, live captions and speaker diarization.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` - Text to speech: spoken audio from text, such as voice-agent replies. → [Streaming TTS quickstart](/tts/getting-started/quickstart).`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`3. **Where is your audio coming from?** (e.g., uploaded files, public URLs, browser microphone, mobile app, Twilio/Telnyx phone numbers, SIP trunks.)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`4. **What language and framework are you using?** (e.g., Python + FastAPI, Node + Next.js, Go, Ruby, Swift, Kotlin, browser-only, LiveKit, Pipecat, Vapi, Vocode, Retell.)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`5. **Do you already have an AssemblyAI API key, or do you need to create one?** (If needed: [assemblyai.com/dashboard/signup](https://www.assemblyai.com/dashboard/signup?utm_source=agent_instructions\u0026utm_medium=ai_agent\u0026utm_campaign=agent_docs).)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`6. **Do you have a data residency requirement?** (US vs EU — this changes the base URL.)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`7. **Anything beyond a plain transcript?** Don't read off a checklist. Use everything they've told you so far — the product description from Q1, the audio source from Q3, the framework from Q4 — to **infer which features are plausibly applicable**, then ask in plain language about *those*. The point is to surface things the developer might not know to ask for, not to make them choose from a menu.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` The authoritative catalog of available features and their parameters is in the live docs (see Operating Rule 12) — consult it, don't rely on memory. Section 3 of this file is a starting reference, not the final word.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` Calibrate to mode and use case. Examples:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` - Customer-support call analytics (pre-recorded) → speaker diarization and PII redaction are almost certainly relevant; sentiment may be; chapters via LLM Gateway often is. Ask about those, not about live-streaming features.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` - Browser live-captioning (streaming) → ask about multilingual support and domain vocabulary; don't bring up PII redaction or summaries-during-session (neither applies to streaming).`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` - Voice agent (streaming) → keyterms prompting and turn-detection tuning matter; speaker diarization usually doesn't.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` - Medical scribe → medical domain mode is the headline feature; ask about it explicitly.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\" Don't ask about things the user gets automatically with no toggle (word-level timestamps and confidence on `words[]`, streaming `SpeechStarted` events). Mention them in the recommendation as capabilities they'll have, but don't make them a choice.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:' If you\\'re confident from context that a feature is needed (e.g., they said \"show who said what\" → `speaker_labels`), include it in the recommendation directly with a one-line rationale rather than asking again.'})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`---`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`## 2. Recommendation template (after discovery)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`Before writing code, post a plan with all of the following. Get explicit approval.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"````\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`## Recommendation`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`**Use case:** \u003cone-sentence summary of what they're building>`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`**Mode:** \u003cpre-recorded / streaming / both>`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`**Region:** \u003cUS or EU base URL>`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`**Model:**`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`- \u003cmodel name> — \u003cone-line rationale>`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`- \u003cfallback model, if applicable>`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`**Endpoints:**`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`- \u003cendpoint 1>`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`- \u003cendpoint 2>`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`**Parameters enabled:** (before filling this in, verify each parameter is supported on the chosen mode + model per Operating Rule 12)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- `param_name`: \u003cvalue> — \u003cwhy>\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`- ...`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`**Auth pattern:**`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`\u003cserver-side key / temp token / proxied uploads — and where the key lives>`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`**Termination \u0026 error handling:**`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`\u003chow streaming sessions are closed; how errors / retries are handled>`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`**Code skeleton:**`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`\u003c2–6 bullet points describing the files/functions you'll generate>`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`**Open questions / assumptions:**`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`\u003canything you inferred that they should confirm>`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`Ready to proceed?`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"````\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`If they say yes, write the code. If they push back on any piece, revise the plan — don't just start coding around objections.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`---`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`## 3. Feature selection guide (agent reference)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`Use this to build the recommendation. Do not dump it on the user.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`| Developer need | Parameter / approach |`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`|---|---|`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"| Speaker diarization | `speaker_labels: true` (pre-recorded, and streaming — streaming adds a `speaker_label` to each Turn event) |\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"| Automatic language detection | `language_detection: true` (pre-recorded; on streaming, adds `language_code` + `language_confidence` to Turn events. It only reports the language; it doesn't change transcription.) |\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:'| Specific language | `language_code: \"es\"` etc. on pre-recorded; on streaming, `language_codes: [\"es\"]` steers the model toward the languages you list |'})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:'| Multilingual / code-switching | `speech_models: [\"assemblyai-stt-3-7\"]` + `prompt` parameter — see [prompting guide](/pre-recorded-audio/prompting) |'})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"| Domain-specific vocabulary | `keyterms_prompt: [...]` (pre-recorded: up to 1,000 terms with AssemblyAI-STT-3.7 / 200 with U2; streaming: up to 100 terms, each ≤50 chars) |\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:'| Medical domain | `domain: \"medical-v1\"` (pre-recorded *and* streaming; supported languages: en, es, de, fr) |'})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:'| Veterinary domain (beta) | `domain: \"veterinary-v1\"` (pre-recorded *and* streaming) |'})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:'| PII redaction in text | `redact_pii: true` + `redact_pii_policies: [...]` + optional `redact_pii_sub: \"hash\" \\\\| \"entity_name\"` |'})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"| PII redaction in audio | `redact_pii_audio: true` (original file must be ≤1 GB; redacted audio URL is available for 24 h) |\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`| Chapters or summaries | Transcribe first, then LLM Gateway (Section 8) |`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"| Word timestamps / confidence | Included by default on `words[]` |\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:'| Webhook delivery (skip polling) | `webhook_url: \"...\"` (Section 7) |'})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`| Custom voice agent (your LLM + AssemblyAI TTS) | Real-time STT + framework integration (Section 10) + [Streaming TTS](/tts/getting-started/quickstart) |`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"| Multilingual streaming | AssemblyAI-STT-3.7-Realtime-Agent or AssemblyAI-STT-3.7-Realtime (both code-switch natively; `language_codes` to steer toward known languages) |\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"| Streaming for notetakers, ambient transcription, live captions or diarization | AssemblyAI-STT-3.7-Realtime (`assemblyai-stt-3-7-realtime`); voice agents use AssemblyAI-STT-3.7-Realtime-Agent (`assemblyai-stt-3-7-realtime-agent`) |\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`---`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`## 4. API overview`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- **REST base URL (US):** `https://api.assemblyai.com`\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- **REST base URL (EU):** `https://api.eu.assemblyai.com`\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- **Streaming WebSocket (Edge, default):** `wss://streaming.assemblyai.com/v3/ws` — auto-routes to the nearest region (Oregon / Virginia / Ireland) for lowest latency\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- **Streaming WebSocket (US data residency):** `wss://streaming.us.assemblyai.com/v3/ws` — data pinned to US\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- **Streaming WebSocket (EU data residency):** `wss://streaming.eu.assemblyai.com/v3/ws` — data pinned to EU\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- **LLM Gateway (US):** `https://llm-gateway.assemblyai.com/v1/chat/completions`\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- **LLM Gateway (EU):** `https://llm-gateway.eu.assemblyai.com/v1/chat/completions` — Claude and Gemini only; OpenAI and Qwen are US-only\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- **Auth header:** `Authorization: YOUR_API_KEY` (no `Bearer`). Same header is used for REST, streaming WS upgrade, temp-token minting, and LLM Gateway\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- **Content type:** `application/json` for submit/poll and LLM Gateway; `application/octet-stream` (raw binary) for `/v2/upload`\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`Core REST endpoints:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:'- `POST /v2/upload` — upload a local file (raw binary body, **not multipart**). Returns `{ \"upload_url\": \"...\" }`. Max 2.2 GB.'})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:'- `POST /v2/transcript` — submit a job. Returns transcript object with `id` and `status: \"queued\"`. Max 5 GB / 10 hours.'})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- `GET /v2/transcript/{id}` — poll. Statuses: `queued`, `processing`, `completed`, `error`. Limited to 60 requests per minute per API key and transcript ID; over the limit returns `429` with a `Retry-After` header. Poll every few seconds, not in a tight loop.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`Streaming:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- `wss://streaming.assemblyai.com/v3/ws?sample_rate=16000\u0026speech_model=assemblyai-stt-3-7-realtime-agent`\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:'- `POST https://token.assemblyai.com/v1/token` with the JSON body `{\"product\": \"streaming\", \"expires_in_seconds\": 60}` — mint a temp token for browser/mobile clients. The token is reusable until `expires_in_seconds` elapses, so mint a fresh one per session. Optional `max_session_duration_seconds` (60–10800, defaults to 3 h) caps the downstream session length. At the cap, the session ends after the current turn or up to 60 s later, and that extra time is billed.'})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`---`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"## 5. `speech_models` semantics\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"`speech_models` on pre-recorded requests is a list of candidate models, not parallel execution. A single transcript is produced by exactly one model.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- **If the list includes `assemblyai-stt-3-7`**, the request goes to AssemblyAI-STT-3.7 for any language it supports (set by `language_code` or detected), wherever it appears in the list. Other models are used only for languages AssemblyAI-STT-3.7 doesn't support. If you set a `language_code` that AssemblyAI-STT-3.7 doesn't support, the request falls back to Universal-2 even if `universal-2` isn't in the list.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- **If the list doesn't include `assemblyai-stt-3-7`**, the first model in the list, from left to right, that supports the language is used.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- **`disfluencies` defaults to `true`** whenever the list includes `assemblyai-stt-3-7` (or `speech_models` is omitted and the request uses `assemblyai-stt-3-7` by default), even if another model is first or transcribes the audio. Set `disfluencies: false` to remove filler words.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:'The recommended setup is `[\"assemblyai-stt-3-7\"]`.'})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"On streaming, the parameter is **singular** (`speech_model=assemblyai-stt-3-7-realtime-agent`) — there is no fallback list. Easy to mix up.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`---`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`## 6. Pre-recorded quick start`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`### SDK (recommended)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`**Python:**`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```python\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`# pip install assemblyai`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`import assemblyai as aai`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`import os`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`aai.settings.api_key = os.environ[\"ASSEMBLYAI_API_KEY\"]`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`config = aai.TranscriptionConfig(`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` speech_models=[\"assemblyai-stt-3-7\"],`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` speaker_labels=True,`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`transcript = aai.Transcriber(config=config).transcribe(\"https://assembly.ai/wildfires.mp3\")`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`# Or a local path: .transcribe(\"./recording.wav\")`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`if transcript.status == aai.TranscriptStatus.error:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` raise RuntimeError(transcript.error)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`print(transcript.text)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`For more test files by scenario (medical, multilingual, meeting, phone, call center, noisy), see [Sample audio files](/samples).`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`**Node/JS:**`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```javascript\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`// npm install assemblyai`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`import { AssemblyAI } from 'assemblyai';`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`const client = new AssemblyAI({ apiKey: process.env.ASSEMBLYAI_API_KEY });`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`const transcript = await client.transcripts.transcribe({`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` audio: 'https://assembly.ai/wildfires.mp3', // or a local file path / Buffer / stream`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` speech_models: [\"assemblyai-stt-3-7\"],`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` speaker_labels: true,`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`});`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`if (transcript.status === 'error') throw new Error(transcript.error);`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`console.log(transcript.text);`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`The SDK handles upload, submit, and polling. You don't need to write the polling loop yourself.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`### Raw HTTP (fallback — use only if SDK isn't an option)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`**Upload a local file** (raw bytes, not multipart):`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```bash\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`curl -X POST https://api.assemblyai.com/v2/upload \\\\`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` -H \"Authorization: $ASSEMBLYAI_API_KEY\" \\\\`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` --data-binary @recording.wav`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`# -> { \"upload_url\": \"https://cdn.assemblyai.com/upload/...\" }`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`**Submit and poll (Python):**`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```python\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`import os, time, requests`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`headers = {\"authorization\": os.environ[\"ASSEMBLYAI_API_KEY\"]}`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`submit = requests.post(`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` \"https://api.assemblyai.com/v2/transcript\",`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` headers=headers,`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` json={`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` \"audio_url\": \"https://assembly.ai/wildfires.mp3\",`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` \"speech_models\": [\"assemblyai-stt-3-7\"],`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` \"speaker_labels\": True,`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` },`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`transcript_id = submit.json()[\"id\"]`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`while True:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` res = requests.get(`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` f\"https://api.assemblyai.com/v2/transcript/{transcript_id}\",`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` headers=headers,`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` ).json()`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` if res[\"status\"] == \"completed\":`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` print(res[\"text\"]); break`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` if res[\"status\"] == \"error\":`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` raise RuntimeError(res[\"error\"])`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` time.sleep(3)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"Common optional params: `speaker_labels`, `language_detection`, `language_code`, `punctuate`, `format_text`, `redact_pii`, `redact_pii_audio`, `keyterms_prompt`, `webhook_url`, `prompt`.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`---`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`## 7. Webhooks (skip polling)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"Provide `webhook_url` on submit; AssemblyAI POSTs when the job finishes:\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```json\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`{ \"transcript_id\": \"5552493-16d8-42d8-8feb-c2a16b56f6e8\", \"status\": \"completed\" }`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`Handler requirements:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`- Return 2xx within **10 seconds**. Otherwise retried up to 10 times, 10s apart. 4xx is not retried.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- On receipt, call `GET /v2/transcript/{id}` to fetch the full result — the webhook payload doesn't include it.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"Optional custom auth on your webhook: set `webhook_auth_header_name` and `webhook_auth_header_value` when submitting.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"**Source IPs** (for allowlists): US `44.238.19.20`, EU `54.220.25.36`.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`**Local dev note:** Webhook URLs must be publicly reachable. Use ngrok, Cloudflare Tunnel, or similar during development.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`---`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`## 8. LLM Gateway (chapters, summaries, custom analysis)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"LLM Gateway replaces both the deprecated transcript params (`auto_chapters`, `summarization`, `summary_model`, `summary_type`) and the legacy **LeMUR** API, which sunset on 2026-03-31. If a developer mentions LeMUR or `transcript_ids`, point them at LLM Gateway and the [migration guide](/llm-gateway/quickstart). Workflow:\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"1. Transcribe normally with `POST /v2/transcript`.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:'2. Once `status == \"completed\"`, POST to LLM Gateway with the transcript text (or paragraphs from `GET /v2/transcript/{id}/paragraphs` for chapter-style output):'})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```http\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`POST https://llm-gateway.assemblyai.com/v1/chat/completions`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`Authorization: YOUR_API_KEY`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`Content-Type: application/json`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`{`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` \"model\": \"claude-sonnet-4-6\",`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` \"messages\": [`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` { \"role\": \"system\", \"content\": \"Produce a 5-bullet summary of the transcript.\" },`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` { \"role\": \"user\", \"content\": \"\u003ctranscript.text here>\" }`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` ],`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` \"max_tokens\": 1000`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`}`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"Model IDs are exact strings — see the [LLM Gateway Overview](/llm-gateway/quickstart) for the current list. Examples: `claude-opus-4-7`, `claude-sonnet-4-6`, `claude-haiku-4-5-20251001`, `gpt-5.2`, `gpt-5.1`, `gpt-4.1`, `gemini-3.5-flash`, `gemini-2.5-pro`, `gemini-2.5-flash`, `qwen3-next-80b-a3b`. `claude-sonnet-4` by itself is **not** valid — always include the version suffix. EU region (`llm-gateway.eu.assemblyai.com`) supports Anthropic and Google only.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"Do not submit with `auto_chapters` and `summarization` both enabled — the API rejects it (`Only one of the following models can be enabled at a time: auto_chapters, summarization.`). But the broader rule is simpler: **don't use either.**\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`---`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`## 9. Streaming — AssemblyAI-STT-3.7-Realtime-Agent`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"**WebSocket (default, Edge Routing):** `wss://streaming.assemblyai.com/v3/ws?sample_rate=16000\u0026speech_model=assemblyai-stt-3-7-realtime-agent`\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"**Model choice:** `assemblyai-stt-3-7-realtime-agent` (AssemblyAI-STT-3.7-Realtime-Agent) for voice agents. `assemblyai-stt-3-7-realtime` (AssemblyAI-STT-3.7-Realtime) for meeting notetakers, ambient transcription, live captions and speaker diarization; it doesn't support Voice Focus, `interruption_delay` has no effect on it, and it rejects `mode: \\\"manual\\\"` with a `3006` error. `universal-3-6-pro` is still supported for existing integrations but isn't recommended for new ones.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"For data residency, swap the host: `streaming.us.assemblyai.com` (US-pinned) or `streaming.eu.assemblyai.com` (EU-pinned). The default host auto-routes to the nearest region.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"**Audio format:** PCM16 signed little-endian, mono, 16 kHz. Binary WebSocket frames, **20–1000 ms per chunk** (50–100 ms recommended), no faster than real-time. Phone audio (`encoding=pcm_mulaw`, `sample_rate=8000`) is sent as-is — don't upsample.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`**Auth:**`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- Server-side: `Authorization` header on the WS upgrade.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- Browser/mobile: mint a short-lived token server-side and pass it as `?token=\u003ctoken>` (no Authorization header).\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`Mint a token:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```bash\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`curl -s -X POST \"https://token.assemblyai.com/v1/token\" \\\\`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` -H \"Authorization: $ASSEMBLYAI_API_KEY\" \\\\`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` -H \"Content-Type: application/json\" \\\\`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` -d '{\"product\": \"streaming\", \"expires_in_seconds\": 60}'`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`# { \"token\": \"...\" }`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"`expires_in_seconds` must be 1–600. A token can start multiple sessions until it expires, so mint a fresh one per session and keep the expiry short.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`### Server messages (JSON)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- `Begin` — `{ type, id, expires_at }`\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- `SpeechStarted` — `{ type, timestamp, confidence }`\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- `Turn` — `{ type, turn_order, end_of_turn, transcript, end_of_turn_confidence, words:[...], utterance }`\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\" - `end_of_turn: false` → partial; `end_of_turn: true` → finalized and formatted. Always read `transcript` for current text.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- `Termination` — `{ type, audio_duration_seconds, session_duration_seconds }`\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`### Client messages`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`- Binary PCM16 frames — audio.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:'- `{ \"type\": \"Terminate\" }` — graceful end. **Always send this when done.**'})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:'- `{ \"type\": \"ForceEndpoint\" }` — force current turn to end.'})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:'- `{ \"type\": \"KeepAlive\" }` — only needed if `inactivity_timeout` is set.'})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:'- `{ \"type\": \"UpdateConfiguration\", \"keyterms_prompt\": [...], \"min_turn_silence\": 100, \"max_turn_silence\": 1000 }` — adjust mid-session.'})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`### SDK (recommended)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`**Python:**`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```python\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`# pip install assemblyai`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`import os`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`from assemblyai.streaming.v3 import (`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` StreamingClient,`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` StreamingClientOptions,`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` StreamingEvents,`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` StreamingParameters,`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` TurnEvent,`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`def on_turn(_, event: TurnEvent):`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` tag = \"FINAL\" if event.end_of_turn else \"partial\"`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` print(f\"{tag}: {event.transcript}\")`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`client = StreamingClient(`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` StreamingClientOptions(api_key=os.environ[\"ASSEMBLYAI_API_KEY\"])`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`client.on(StreamingEvents.Turn, on_turn)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`client.connect(StreamingParameters(sample_rate=16000, speech_model=\"assemblyai-stt-3-7-realtime-agent\"))`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`# Feed 16 kHz mono PCM16 chunks (50–1000ms each) via client.stream(chunk)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`# When finished:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`client.disconnect(terminate=True) # sends Terminate and closes cleanly`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`**Node/JS:**`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```javascript\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`// npm install assemblyai`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`import { AssemblyAI } from 'assemblyai';`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`const client = new AssemblyAI({ apiKey: process.env.ASSEMBLYAI_API_KEY });`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`const rt = client.streaming.transcriber({`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` sampleRate: 16000,`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` speechModel: 'assemblyai-stt-3-7-realtime-agent',`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`});`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`rt.on('turn', (turn) => {`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` const tag = turn.end_of_turn ? 'FINAL' : 'partial';`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\" console.log(`${tag}: ${turn.transcript}`);\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`});`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`rt.on('error', (err) => console.error(err));`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`await rt.connect();`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`// rt.sendAudio(pcm16Buffer) for each 50–1000ms chunk`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`// When done:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`await rt.close(); // sends Terminate and closes`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`### Raw WebSocket (fallback)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"**Node.js (`ws`):**\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```javascript\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`import WebSocket from 'ws';`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`const ws = new WebSocket(`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` 'wss://streaming.assemblyai.com/v3/ws?sample_rate=16000\u0026speech_model=assemblyai-stt-3-7-realtime-agent',`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` { headers: { authorization: process.env.ASSEMBLYAI_API_KEY } },`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`);`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`ws.on('open', () => {`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` // Feed PCM16 16kHz mono chunks here, 50–1000ms each.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` // Example: audioStream.on('data', (chunk) => ws.send(chunk));`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`});`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`ws.on('message', (raw) => {`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` const msg = JSON.parse(raw.toString());`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` if (msg.type === 'Turn') {`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\" console.log(msg.end_of_turn ? `FINAL: ${msg.transcript}` : `partial: ${msg.transcript}`);\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` }`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`});`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`function stop() {`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` ws.send(JSON.stringify({ type: 'Terminate' })); // required!`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`}`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"**Python (`websockets`):**\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```python\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`import asyncio, json, os, websockets`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`URL = \"wss://streaming.assemblyai.com/v3/ws?sample_rate=16000\u0026speech_model=assemblyai-stt-3-7-realtime-agent\"`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`async def run(audio_source):`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` \"\"\"audio_source: async iterator yielding 50–1000ms PCM16 chunks at 16kHz mono.\"\"\"`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` async with websockets.connect(`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` URL,`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` additional_headers={\"Authorization\": os.environ[\"ASSEMBLYAI_API_KEY\"]},`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` ) as ws:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` async def send_audio():`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` async for chunk in audio_source:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` await ws.send(chunk)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` await ws.send(json.dumps({\"type\": \"Terminate\"}))`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` async def recv_loop():`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` async for raw in ws:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` msg = json.loads(raw)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` if msg[\"type\"] == \"Turn\":`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` tag = \"FINAL\" if msg[\"end_of_turn\"] else \"partial\"`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` print(f\"{tag}: {msg['transcript']}\")`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` elif msg[\"type\"] == \"Termination\":`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` return`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` await asyncio.gather(send_audio(), recv_loop())`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`# asyncio.run(run(my_audio_iterator()))`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`---`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`## 10. Voice Agent framework configs (real-time STT + your own pipeline)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`This section is for developers who are wiring AssemblyAI Real-time STT into LiveKit, Pipecat, Vapi, Vocode, Retell, or similar, and bringing their own LLM. For the agent's voice, use AssemblyAI [Streaming TTS](/tts/getting-started/quickstart), which has plugins for [LiveKit](/tts/livekit) and [Pipecat](/tts/pipecat).`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`The defaults will not be good enough. Common tuning:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- **`keyterms_prompt`** — pass proper nouns, product names, and domain terms. For dynamic values (usernames, order IDs), update mid-session via `UpdateConfiguration`.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- **Turn silence bounds** — `min_turn_silence` and `max_turn_silence` (ms). Lower values fire end-of-turn faster but risk cutting speakers off. Higher values reduce false finalizations. Form-filling and dictation use cases often want wider windows.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:'- **Multilingual** — Streaming code-switches natively. Pass `language_codes` (for example `[\"es\"]`) to steer the model toward known languages.'})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- **Barge-in / false SpeechStarted** — ambient noise, TTS bleed-through, and PSTN echo can cause spurious `SpeechStarted` events. If the agent is interrupting itself, look here first. Framework-level knobs (e.g., LiveKit's `min_interruption_duration`) often complement, not replace, server-side tuning.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- **Phone audio** — 8 kHz mu-law (`pcm_mulaw` at 8000 Hz) should be sent as-is, not upsampled to 16 kHz. Upsampling degrades accuracy.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`When the developer names one of these frameworks, ask about their specific turn-taking and interruption requirements before defaulting.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`---`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`## 11. Browser patterns`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`**Never put the API key in client code.**`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`### Pre-recorded — proxy upload + submit through your server`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```javascript\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`// Next.js route handler (server)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`export async function POST(request) {`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` const incoming = await request.formData();`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` const file = incoming.get('file'); // Blob`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` const upload = await fetch('https://api.assemblyai.com/v2/upload', {`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` method: 'POST',`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` headers: { authorization: process.env.ASSEMBLYAI_API_KEY },`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` body: file.stream(),`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` duplex: 'half',`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` });`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` const { upload_url } = await upload.json();`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` const submit = await fetch('https://api.assemblyai.com/v2/transcript', {`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` method: 'POST',`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` headers: {`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` authorization: process.env.ASSEMBLYAI_API_KEY,`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` 'content-type': 'application/json',`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` },`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` body: JSON.stringify({`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` audio_url: upload_url,`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` speech_models: ['assemblyai-stt-3-7'],`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` }),`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` });`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` return Response.json(await submit.json());`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`}`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`### Streaming — server mints a temp token, client connects directly`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```javascript\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`// Server`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`export async function GET() {`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` const res = await fetch('https://token.assemblyai.com/v1/token', {`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` method: 'POST',`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` headers: {`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` authorization: process.env.ASSEMBLYAI_API_KEY,`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` 'content-type': 'application/json',`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` },`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` body: JSON.stringify({ product: 'streaming', expires_in_seconds: 60 }),`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` });`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` return Response.json(await res.json()); // { token }`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`}`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```javascript\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`// Client`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`const { token } = await fetch('/api/aai-token').then((r) => r.json());`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`const ws = new WebSocket(`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\" `wss://streaming.assemblyai.com/v3/ws?sample_rate=16000\u0026speech_model=assemblyai-stt-3-7-realtime-agent\u0026token=${token}`,\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`);`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`ws.onmessage = (e) => {`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` const msg = JSON.parse(e.data);`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` if (msg.type === 'Turn') console.log(msg.transcript, msg.end_of_turn);`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`};`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"```\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`### Capturing mic audio in the browser`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"`MediaRecorder` does not emit PCM16. You need an `AudioWorklet` (preferred) or `ScriptProcessorNode` to:\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`1. Capture raw Float32 samples.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`2. Downsample to 16 kHz.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`3. Convert Float32 → Int16.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`4. Send each ~50 ms chunk as a binary WS frame.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`Reference: [AssemblyAI realtime-transcription-browser-js-example](https://github.com/AssemblyAI/realtime-transcription-browser-js-example).`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`---`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`## 12. Not supported / out of scope`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`If a developer asks for any of these, say so directly and propose the closest supported alternative. Do not improvise.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- **Real-time translation.** There is no standalone translation model. For live audio, use [real-time translation](/streaming/real-time-translation): pass an `llm_gateway` connection parameter with a translation prompt, and each completed turn comes back translated on the same WebSocket as an `LLMGatewayResponse` message.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`- **On-device / offline STT.** Cloud API only.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- **Speaker identification (matching voices to known people).** `speaker_labels` does diarization (Speaker A, B, C) but does not recognize specific individuals. For pre-recorded (async) audio, add the Speech Understanding [Speaker Identification](/speech-understanding/speaker-identification) task (`speech_understanding.request.speaker_identification` with `speaker_labels: true`) to replace the generic labels with the names or roles you provide. Speaker Identification isn't available for streaming or other products yet.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- **Voice activity detection as a standalone product.** VAD is internal to the streaming pipeline and surfaced via `SpeechStarted` / turn events, not exposed separately.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`---`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`## 13. Error handling`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`For the most common errors across all APIs, with fixes, and links to each product's full error and limit reference, see [Errors and limits](/errors).`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`### REST (pre-recorded)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- **401** — Missing/invalid Authorization, disabled account, or insufficient balance. Double-check there's no `Bearer` prefix.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:'- **Transcript `status: \"error\"`** — Read the `error` field on `GET /v2/transcript/{id}`.'})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- **Retries** — Exponential backoff on 5xx. For 429, respect the `Retry-After` header.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- **Limits** — `/v2/upload` max 2.2 GB; `/v2/transcript` max 5 GB / 10 hr per file.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`- **Scoping** — An API key can only transcribe files uploaded under the same project.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`### Streaming — handshake`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- **HTTP 410** — The old `v2` streaming endpoint is deprecated. Upgrade to `/v3/ws`. This is an HTTP status on the upgrade request, not a WebSocket close code.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`### Streaming — WebSocket close codes`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`| Code | Meaning |`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`|------|---------|`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"| `1008` | Unauthorized: missing or invalid key or token, expired temporary token, insufficient funds, or disabled key. Don't retry until fixed; tell it apart from the rate limit by the `Error` message's `error` text |\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"| `1008` | Too many new sessions this minute (`error` text `Unauthorized Connection: Too many concurrent sessions`); back off and retry next minute |\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"| `3005` | Session cancelled (server-side error) |\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"| `3006` | Invalid message type / invalid JSON |\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"| `3007` | Audio chunk outside 20–1000 ms, or sent faster than real-time |\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"| `3008` | Session expired (3-hour cap) |\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`### Streaming gotchas`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- `speech_model` (streaming, singular) vs `speech_models` (pre-recorded, plural). Don't mix up.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- On streaming, `language_codes` steers the model toward the languages you list.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:'- Always send `{ \"type\": \"Terminate\" }` when finished. An abandoned session stays billable until the 3-hour cap (`3008`).'})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- Chunk size matters: frames outside 20–1000 ms will close the socket with `3007`.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`---`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`## 14. Quick-reference gotchas`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- No `Bearer` prefix on the Authorization header.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:'- `speech_models` is **optional** on pre-recorded submits — for new accounts created on or after November 17, 2026 it defaults to `[\"assemblyai-stt-3-7\"]`, and when provided, `assemblyai-stt-3-7` takes priority wherever it appears in the list (see section 5).'})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- `/v2/upload` takes **raw binary**, not multipart.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`- Webhook handlers must return 2xx in ≤10 seconds.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`- Local webhook development needs a public tunnel (ngrok, Cloudflare Tunnel).`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`- Browser code never holds the API key. Proxy uploads, or mint temp tokens for streaming.`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- Always `Terminate` streaming sessions.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- Don't use `auto_chapters`, `summarization`, `summary_model`, `summary_type`. Use LLM Gateway.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:'- Medical mode is `domain: \"medical-v1\"` (pre-recorded body param / streaming query param). The legacy `medical_mode` flag is **not** the right name.'})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:'- Veterinary mode (beta) is `domain: \"veterinary-v1\"` (pre-recorded body param / streaming query param).'})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- LLM Gateway model IDs are exact and versioned (e.g., `claude-sonnet-4-6`, `gpt-5.2`, `gemini-2.5-pro`). Shorthand like `claude-sonnet-4` is invalid.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- Phone audio stays at native 8 kHz mu-law (`encoding=pcm_mulaw`) — don't upsample.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- EU customers use `api.eu.assemblyai.com`, `streaming.eu.assemblyai.com`, and `llm-gateway.eu.assemblyai.com`. The default streaming host (`streaming.assemblyai.com`) is **Edge Routing**, not US-pinned — use `streaming.us.assemblyai.com` if you need data residency guarantees on the US side.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:'- Speech-model values are **raw strings** in the SDKs (`\"assemblyai-stt-3-7\"`, `\"universal-2\"`). Enum aliases like `aai.SpeechModel.universal_3_pro` do **not** exist — agents that hallucinate them produce code that imports cleanly and fails at runtime.'})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:\"- LeMUR has fully sunset (2026-03-31). Don't generate code that calls LeMUR endpoints or passes `transcript_ids` to a chat-completions API — use LLM Gateway with the transcript text in `messages` instead.\"})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})})]})})})}),`\n`,_jsxs(_components.p,{children:[`The same prompt is published at `,_jsx(_components.a,{href:`/agent-instructions.md`,children:`agent-instructions.md`}),` if you’d rather link to it than paste the whole thing.`]}),`\n`,_jsx(Heading,{level:`2`,id:`docs-mcp-server`,children:`Docs MCP server`}),`\n`,_jsx(Note,{children:_jsxs(_components.p,{children:[`Want an AI app to transcribe audio for you, not write code? Use the `,_jsx(_components.a,{href:`/mcp-server`,children:`AssemblyAI MCP server`}),` instead. It signs in to your account and runs the API tasks itself.`]})}),`\n`,_jsx(_components.p,{children:`For on-demand lookups during a session — useful when the agent is iterating on a feature and needs to check several pages mid-task — connect the AssemblyAI docs MCP server:`}),`\n`,_jsx(CodeBlock,{numberOfLines:`2`,language:`text`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`text`,children:_jsxs(_components.code,{language:`text`,numberOfLines:`2`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`https://www.assemblyai.com/docs/mcp`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})})]})})}),`\n`,_jsx(_components.p,{children:_jsx(_components.strong,{children:`Claude Code:`})}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`1`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`1`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#DCDCAA`},children:`claude`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` mcp`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` add`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` assemblyai-docs`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:` --transport`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` http`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` https://www.assemblyai.com/docs/mcp`})]}),`\n`]})})}),`\n`,_jsxs(_components.p,{children:[_jsx(_components.strong,{children:`Cursor`}),` (`,_jsx(_components.code,{children:`.cursor/mcp.json`}),`):`]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`7`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`7`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"mcpServers\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: {`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"assemblyai-docs\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: {`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"url\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"https://www.assemblyai.com/docs/mcp\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` }`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` }`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})}),`\n`,_jsxs(_components.p,{children:[`Any MCP client that supports `,_jsx(_components.a,{href:`https://modelcontextprotocol.io/specification/2025-03-26/basic/transports#streamable-http`,children:`Streamable HTTP transport`}),` can connect. The server exposes tools to search and fetch AssemblyAI documentation pages on demand.`]}),`\n`,_jsx(Heading,{level:`2`,id:`claude-code-skill`,children:`Claude Code skill`}),`\n`,_jsxs(_components.p,{children:[`The `,_jsx(_components.a,{href:`https://github.com/AssemblyAI/assemblyai-skill`,children:`AssemblyAI skill`}),` gives Claude Code curated instructions and context for the Python and JavaScript SDKs, streaming, voice agents, audio intelligence, and more. Install it with the `,_jsx(_components.a,{href:`https://github.com/vercel-labs/skills`,children:`skills CLI`}),`, which also works with Cursor, Copilot, Codex, and 60+ other AI coding agents:`]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`1`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`1`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#DCDCAA`},children:`npx`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` skills`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` add`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` AssemblyAI/assemblyai-skill`})]}),`\n`]})})}),`\n`,_jsxs(_components.p,{children:[`The CLI asks which agents to install it for. Run `,_jsx(_components.code,{children:`npx skills check`}),` later to see whether an update is available.`]}),`\n`,_jsx(Heading,{level:`2`,id:`tips-for-best-results`,children:`Tips for best results`}),`\n`,_jsxs(_components.ul,{children:[`\n`,_jsxs(_components.li,{children:[_jsx(_components.strong,{children:`Be specific about the SDK`}),` — say “use the AssemblyAI Python SDK” or “use the JavaScript SDK” rather than “use AssemblyAI”.`]}),`\n`,_jsxs(_components.li,{children:[_jsx(_components.strong,{children:`Reference the model`}),` — specify `,_jsx(_components.code,{children:`assemblyai-stt-3-7`}),` for pre-recorded or `,_jsx(_components.code,{children:`assemblyai-stt-3-7-realtime-agent`}),` for streaming to avoid outdated model names.`]}),`\n`,_jsxs(_components.li,{children:[_jsx(_components.strong,{children:`Set your API key as an environment variable`}),` — `,_jsx(_components.code,{children:`export ASSEMBLYAI_API_KEY=your_key`}),` so the agent can reference it naturally.`]}),`\n`]})]})}function MDXContent(props={}){const{wrapper:MDXLayout}={..._provideComponents(),...props.components};return MDXLayout?_jsx(MDXLayout,{...props,children:_jsx(_createMdxContent,{...props})}):_createMdxContent(props)}return{default:MDXContent};function _missingMdxReference(id,component){throw new Error(`Expected `+(component?`component`:`object`)+\" `\"+id+\"` to be defined: you likely forgot to import, pass, or provide it.\")}","frontmatter":{},"scope":{"config":{},"pageMetadata":{"title":"Build with AI agents","og.description":"Build with AssemblyAI using Cursor, Claude Code, Copilot, Devin, and other AI coding assistants. Copy the integration prompt, pin the URL in your agent's project instructions, connect the docs MCP server, or install the Claude Code skill.","slug":"/coding-agent-prompts","description":null,"href":"/coding-agent-prompts"}}},"mdxSourceWithNoJs":null},"mdxExtracts":{"tableOfContents":[{"title":"Start here","slug":"start-here","depth":2,"children":[]},{"title":"Compare all four methods","slug":"compare-all-four-methods","depth":2,"children":[]},{"title":"Pin the URL in your agent instructions","slug":"pin-the-url-in-your-agent-instructions","depth":2,"children":[]},{"title":"Copy the integration prompt","slug":"copy-the-integration-prompt","depth":2,"children":[]},{"title":"Docs MCP server","slug":"docs-mcp-server","depth":2,"children":[]},{"title":"Claude Code skill","slug":"claude-code-skill","depth":2,"children":[]},{"title":"Tips for best results","slug":"tips-for-best-results","depth":2,"children":[]}],"codeExamples":{}},"jsLinks":["/docs/_mintlify/custom/assemblyai-fff14a6f/c705301dcef66bf6/rudderstack.js","/docs/_mintlify/custom/assemblyai-fff14a6f/700d2574de973df9/chat-widget.js","/docs/_mintlify/custom/assemblyai-fff14a6f/33c4022d4e30e01c/ketch.js"],"subdomain":"assemblyai-fff14a6f","siteName":"AssemblyAI","title":"Build with AI agents - AssemblyAI","navMaps":{"firstHrefInVersion":{},"firstHrefInLanguage":{}},"lastModified":"2026-11-17T14:23:30.348Z","originalFileLocation":"coding-agent-prompts.mdx"},"headHtml":"\u003cscript data-mint-head>(function(){var d=document.documentElement,p=null;if(!p){try{p=localStorage.getItem(\"isDarkMode\")}catch(e){}if(p==='true')p='dark';else if(p==='false')p='light';if(p!=='light'\u0026\u0026p!=='dark'\u0026\u0026p!=='system')p=\"system\"}var r=p==='system'?(matchMedia(\"(prefers-color-scheme: dark)\").matches?'dark':'light'):p;d.classList.remove(r==='dark'?'light':'dark');d.classList.add(r);d.setAttribute(\"data-theme-preference\",p)})();\u003c/script>\u003cmeta data-mint-head name=\"application-name\" content=\"AssemblyAI\">\u003cmeta data-mint-head name=\"generator\" content=\"Mintlify\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/docs/mintlify-assets/_mintlify/favicons/assemblyai-fff14a6f/IP3FX8MjuVkHah_W/_generated/favicon/android-chrome-192x192.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"apple-touch-icon\" type=\"image/png\" sizes=\"180x180\" href=\"/docs/mintlify-assets/_mintlify/favicons/assemblyai-fff14a6f/IP3FX8MjuVkHah_W/_generated/favicon/apple-touch-icon.png\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/docs/mintlify-assets/_mintlify/favicons/assemblyai-fff14a6f/IP3FX8MjuVkHah_W/_generated/favicon/favicon-16x16.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/docs/mintlify-assets/_mintlify/favicons/assemblyai-fff14a6f/IP3FX8MjuVkHah_W/_generated/favicon/favicon-32x32.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/docs/mintlify-assets/_mintlify/favicons/assemblyai-fff14a6f/IP3FX8MjuVkHah_W/_generated/favicon/favicon.ico\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/docs/mintlify-assets/_mintlify/favicons/assemblyai-fff14a6f/IP3FX8MjuVkHah_W/_generated/favicon-dark/android-chrome-192x192.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/docs/mintlify-assets/_mintlify/favicons/assemblyai-fff14a6f/IP3FX8MjuVkHah_W/_generated/favicon-dark/favicon-16x16.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/docs/mintlify-assets/_mintlify/favicons/assemblyai-fff14a6f/IP3FX8MjuVkHah_W/_generated/favicon-dark/favicon-32x32.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/docs/mintlify-assets/_mintlify/favicons/assemblyai-fff14a6f/IP3FX8MjuVkHah_W/_generated/favicon-dark/favicon.ico\" media=\"(prefers-color-scheme: dark)\">\u003cmeta data-mint-head property=\"og:title\" content=\"Build with AI agents - AssemblyAI\">\u003cmeta data-mint-head property=\"og:site_name\" content=\"AssemblyAI\">\u003cmeta data-mint-head property=\"og:url\" content=\"https://www.assemblyai.com/docs/coding-agent-prompts\">\u003cmeta data-mint-head property=\"og:type\" content=\"website\">\u003cmeta data-mint-head property=\"og:image\" content=\"https://assemblyai-fff14a6f.mintlify.site/docs/_mintlify/api/og/v2?division=Getting+started\u0026amp;title=Build+with+AI+agents\u0026amp;theme=7f291be5810d1f7c5136ed41\u0026amp;signature=4ae9fb39f4eea1d029498d8cdbab8b2c205a0b23056ee2ae552e7b45ca658b8d\">\u003cmeta data-mint-head property=\"og:image:width\" content=\"1200\">\u003cmeta data-mint-head property=\"og:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"twitter:card\" content=\"summary_large_image\">\u003cmeta data-mint-head name=\"twitter:title\" content=\"Build with AI agents - AssemblyAI\">\u003cmeta data-mint-head name=\"twitter:image\" content=\"https://assemblyai-fff14a6f.mintlify.site/docs/_mintlify/api/og/v2?division=Getting+started\u0026amp;title=Build+with+AI+agents\u0026amp;theme=7f291be5810d1f7c5136ed41\u0026amp;signature=4ae9fb39f4eea1d029498d8cdbab8b2c205a0b23056ee2ae552e7b45ca658b8d\">\u003cmeta data-mint-head name=\"twitter:image:width\" content=\"1200\">\u003cmeta data-mint-head name=\"twitter:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"msapplication-config\" content=\"/docs/mintlify-assets/_mintlify/favicons/assemblyai-fff14a6f/IP3FX8MjuVkHah_W/_generated/favicon/browserconfig.xml\">\u003cmeta data-mint-head name=\"apple-mobile-web-app-title\" content=\"AssemblyAI\">\u003cmeta data-mint-head name=\"msapplication-TileColor\" content=\"#614fd2\">\u003clink data-mint-head rel=\"canonical\" href=\"https://www.assemblyai.com/docs/coding-agent-prompts\">\u003clink data-mint-head rel=\"alternate\" type=\"application/xml\" href=\"/docs/sitemap.xml\">\u003clink data-mint-head rel=\"alternate\" type=\"text/markdown\" href=\"/docs/coding-agent-prompts.md\">\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https://www.assemblyai.com/#organization\",\"name\":\"AssemblyAI\",\"url\":\"https://www.assemblyai.com\",\"logo\":{\"@type\":\"ImageObject\",\"url\":\"https://mintcdn.com/assemblyai-fff14a6f/vMuNaI_PbWUbVeV0/assets/assemblyai-lockup-primary.svg?fit=max\u0026auto=format\u0026n=vMuNaI_PbWUbVeV0\u0026q=85\u0026s=808d2a677e2536423045d44b3f330ac2\"}},{\"@type\":\"WebSite\",\"@id\":\"https://www.assemblyai.com/docs#website\",\"name\":\"AssemblyAI\",\"url\":\"https://www.assemblyai.com/docs\",\"publisher\":{\"@id\":\"https://www.assemblyai.com/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https://www.assemblyai.com/docs/coding-agent-prompts#webpage\",\"url\":\"https://www.assemblyai.com/docs/coding-agent-prompts\",\"name\":\"Build with AI agents\",\"dateModified\":\"2026-11-17T14:23:30.348Z\",\"isPartOf\":{\"@id\":\"https://www.assemblyai.com/docs#website\"},\"breadcrumb\":{\"@id\":\"https://www.assemblyai.com/docs/coding-agent-prompts#breadcrumb\"}},{\"@type\":\"BreadcrumbList\",\"@id\":\"https://www.assemblyai.com/docs/coding-agent-prompts#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Getting started\",\"item\":\"https://www.assemblyai.com/docs\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Build with AI agents\",\"item\":\"https://www.assemblyai.com/docs/coding-agent-prompts\"}]},{\"@type\":[\"Article\",\"TechArticle\"],\"@id\":\"https://www.assemblyai.com/docs/coding-agent-prompts#article\",\"headline\":\"Build with AI agents\",\"name\":\"Build with AI agents\",\"url\":\"https://www.assemblyai.com/docs/coding-agent-prompts\",\"mainEntityOfPage\":{\"@id\":\"https://www.assemblyai.com/docs/coding-agent-prompts#webpage\"},\"dateModified\":\"2026-11-17T14:23:30.348Z\",\"publisher\":{\"@id\":\"https://www.assemblyai.com/#organization\"},\"isPartOf\":{\"@id\":\"https://www.assemblyai.com/docs#website\"}}]}\u003c/script>","agentPointerHref":"/docs/coding-agent-prompts.md","siteName":"AssemblyAI","nav":{"global":null,"tabs":[{"tab":"Overview","groups":[{"group":"Getting started","pages":[{"title":"AssemblyAI Documentation","sidebarTitle":"Overview","href":""},{"title":"Build with SDKs","description":null,"href":"/getting-started/sdks"},{"title":"Build with AI agents","href":"/coding-agent-prompts"},{"title":"Use AssemblyAI in AI apps with the MCP server","description":"Connect Claude, Cursor or any MCP client to your AssemblyAI account so it can transcribe, summarize and analyze audio for you.","sidebarTitle":"AssemblyAI MCP server","href":"/mcp-server"},{"title":"Build with integrations","href":"/integrations"}]},{"group":"Build \u0026 Evaluate","pages":[{"title":"Models","href":"/getting-started/models"},{"title":"Evaluations","sidebarTitle":"Evaluate model accuracy","href":"/evaluations"},{"group":"End-to-end examples","root":{"title":"End-to-end examples","href":"/getting-started/end-to-end-examples"},"pages":[{"title":"Meeting notetaker","href":"/getting-started/end-to-end-examples/meeting-notetaker"},{"title":"Sales call intelligence","href":"/getting-started/end-to-end-examples/sales-call-intelligence"},{"title":"Medical scribe","href":"/getting-started/end-to-end-examples/medical-scribe"},{"title":"Content repurposing","href":"/getting-started/end-to-end-examples/content-repurposing"},{"title":"Real-time meeting assistant","href":"/getting-started/end-to-end-examples/real-time-meeting-assistant"},{"title":"Real-time live captioner","href":"/getting-started/end-to-end-examples/real-time-live-captioner"},{"title":"Real-time voice AI on ESP32","href":"/getting-started/end-to-end-examples/realtime-esp32"}]},{"title":"Speech model versions and deprecations","sidebarTitle":"Speech model versions and deprecations","href":"/getting-started/model-versions-and-deprecations"}]},{"group":"Administration","pages":[{"title":"Account Management","sidebarTitle":"Account Management","href":"/account-management"},{"title":"Temporary tokens","sidebarTitle":"Temporary tokens","href":"/temporary-tokens"},{"title":"Single Sign-On (SSO)","sidebarTitle":"Single Sign-On","href":"/sso"},{"title":"Billing and Pricing","sidebarTitle":"Billing and Pricing","href":"/billing-and-pricing"}]},{"group":"Trust \u0026 security","pages":[{"title":"Trust center","url":"https://assembly.ai/trust","href":"/nav-links/trust-center"},{"title":"Security overview","url":"https://www.assemblyai.com/security","href":"/nav-links/security-overview"},{"title":"Data Controls","sidebarTitle":"Data Controls","href":"/data-controls"},{"title":"Data retention and model training","href":"/data-retention-and-model-training"}]}]},{"tab":"Pre-recorded STT","pages":[{"title":"Pre-recorded STT","href":"/pre-recorded-audio/getting-started/transcribe-an-audio-file"}]},{"tab":"Real-time STT","pages":[{"title":"Real-time STT","href":"/streaming/getting-started/transcribe-streaming-audio"}]},{"tab":"Sync STT","pages":[{"title":"Sync STT","href":"/sync-stt/getting-started/quickstart"}]},{"tab":"Dictation API","pages":[{"title":"Dictation API","href":"/dictation"}]},{"tab":"Text to Speech","pages":[{"title":"Text to Speech","href":"/tts/getting-started/quickstart"}]},{"tab":"Voice Agent API","hidden":true,"pages":[{"title":"Voice Agent API","href":"/voice-agents/voice-agent-api"}]},{"tab":"Speech Understanding","pages":[{"title":"Speech Understanding","href":"/speech-understanding/getting-started"}]},{"tab":"LLM Gateway","pages":[{"title":"LLM Gateway","href":"/llm-gateway/quickstart"}]},{"tab":"FAQ","pages":[{"title":"FAQ","href":"/faq"}]}]},"basePath":"","title":"Build with AI agents - AssemblyAI","deploymentId":"dpl_2MqcLy8smQrrovUehgcZEsoo1tJX","buildId":"dpl_2MqcLy8smQrrovUehgcZEsoo1tJX","dynamics":["10e4515597"],"chunks":["BannerMarkdown-c3oKHz6v.js","dist-CLdzWocy.js","lib-DMeTj1gb.js","lib-B8HbPgcC.js","katex-Dbtb6Wk7.js","html-void-elements-mCSPHezU.js","marked.esm-CZWythHk.js"]}