Saved prompts

Overview

Saved prompts let you store a prompt once and run it by name. A saved prompt is a complete chat completions request: the model, the messages, and any other settings. To run it, send a chat completion request with aai_prompt set to the prompt’s name. You can fill in {{ tag }} placeholders with different values on each request. Use saved prompts to:

Keep a prompt out of your application code, and change it without a deploy.
Reuse the same prompt, model and settings across services.
Run one prompt over many transcripts with transcript_id.

Saved prompts belong to your account. Names are unique within your account, and other accounts can’t read, change or use your prompts.

Create a saved prompt

Send a POST request to /v1/prompts with a name and a prompt. The prompt is a JSON string of a chat completions request body.

curl -X POST "https://llm-gateway.assemblyai.com/v1/prompts" \
  -H "Authorization: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "call-summary",
    "prompt": "{\"model\": \"gemini-2.5-flash-lite\", \"max_tokens\": 500, \"messages\": [{\"role\": \"system\", \"content\": \"You summarize calls for {{ team }}.\"}, {\"role\": \"user\", \"content\": \"Summarize this call: {{ transcript }}\"}]}"
  }'

The response is the saved prompt, with 201 Created:

{
  "name": "call-summary",
  "prompt": "{\"model\": \"gemini-2.5-flash-lite\", ...}",
  "created_at": "2026-10-28T16:04:11.512Z",
  "updated_at": "2026-10-28T16:04:11.512Z"
}

The name and prompt must meet these limits:

Field	Limit
`name`	Required. Up to 128 characters. Starts with a letter or number, then letters, numbers, `.`, `_` and `-`. Unique within your account.
`prompt`	Required. Up to 50 KiB. A JSON string that is a valid chat completions request, including a supported `model`.

A request that breaks these limits returns 400, with each problem listed in metadata.errors. Creating a name your account already uses returns 409.

Run a saved prompt

Send a request to /v1/chat/completions with aai_prompt set to the prompt’s name.

curl -X POST "https://llm-gateway.assemblyai.com/v1/chat/completions" \
  -H "Authorization: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "aai_prompt": "call-summary",
    "prompt_replacements": {"team": "Support"},
    "transcript_id": "065a71ac-dc3e-4e38-9374-e54c0bea564f"
  }'

The stored prompt replaces the request, so the model, messages and every other setting come from the prompt. The response is a normal chat completion.

Fields you send alongside aai_prompt are ignored, with these exceptions: prompt_replacements and transcript_id. Those apply to the stored prompt. model is optional when aai_prompt is set.

If your account has no prompt with that name, the request returns 404.

Fill in tags

Put {{ name }} placeholders in the messages of a saved prompt, then set prompt_replacements to an object of values to substitute:

A tag is {{, a space, the key, a space, then }}. Variants without the spaces, like {{team}}, aren’t replaced.
Every occurrence of a tag is replaced, in message text, in each part of a message with multiple content parts, and in prompt.
Tags are also replaced in the prompt’s fallbacks, one level deep. Tags in fallbacks nested inside a fallback are left as they are.
Replacement runs once. If a value contains a tag, that tag isn’t replaced in turn.
A tag with no matching key is left as it is.

prompt_replacements requires aai_prompt. Sending it without aai_prompt returns 400.

Use a transcript

The transcript tag is reserved. Put {{ transcript }} in a saved prompt and pass transcript_id when you run it, and LLM Gateway injects the transcript text as described in Inject a transcript by ID. Without a transcript_id, the tag is left as it is. Setting transcript in prompt_replacements returns 400.

Get, update and delete a saved prompt

Each of these takes the prompt’s name in the path.

# Get
curl "https://llm-gateway.assemblyai.com/v1/prompts/call-summary" \
  -H "Authorization: YOUR_API_KEY"

# Update: replaces the prompt, keeps the name
curl -X PUT "https://llm-gateway.assemblyai.com/v1/prompts/call-summary" \
  -H "Authorization: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"prompt": "{\"model\": \"claude-sonnet-4-6\", \"messages\": [{\"role\": \"user\", \"content\": \"Summarize: {{ transcript }}\"}]}"}'

# Delete
curl -X DELETE "https://llm-gateway.assemblyai.com/v1/prompts/call-summary" \
  -H "Authorization: YOUR_API_KEY"

Request	Success	Errors
`GET`	`200` with the saved prompt	`404` if your account has no prompt with that name
`PUT`	`200` with the updated prompt	`400` for an invalid prompt, `404` if there is no such prompt
`DELETE`	`204` with no body	`404` if your account has no prompt with that name

A prompt that belongs to another account returns 404, the same as one that doesn’t exist.

Errors

Status	When
`400`	An invalid name or prompt, `prompt_replacements` without `aai_prompt`, or `transcript` in `prompt_replacements`.
`404`	The saved prompt doesn’t exist in your account, or the `transcript_id` doesn’t exist.
`409`	Creating a prompt with a name your account already uses.

See the API reference for the full request and response of each endpoint.

Getting started

Features

API reference

Core concepts

Voice AI Guides

Overview

Create a saved prompt

Run a saved prompt

Fill in tags

Use a transcript

Get, update and delete a saved prompt

Errors

​Overview

​Create a saved prompt

​Run a saved prompt

​Fill in tags

​Use a transcript

​Get, update and delete a saved prompt

​Errors

Overview

Create a saved prompt

Run a saved prompt

Fill in tags

Use a transcript

Get, update and delete a saved prompt

Errors