Skip to main content

Overview

Saved prompts let you store a prompt once and run it by name. A saved prompt is a complete chat completions request: the model, the messages, and any other settings. To run it, send a chat completion request with aai_prompt set to the prompt’s name. You can fill in {{ tag }} placeholders with different values on each request. Use saved prompts to:
  • Keep a prompt out of your application code, and change it without a deploy.
  • Reuse the same prompt, model and settings across services.
  • Run one prompt over many transcripts with transcript_id.
Saved prompts belong to your account. Names are unique within your account, and other accounts can’t read, change or use your prompts. Saved prompts and prompt templates are stored in your account until you delete them. They have no expiry or TTL. Deleting a saved prompt removes it permanently, and updating one overwrites it in place. Values filled into a saved prompt at request time, such as prompt_replacements or a transcript injected with transcript_id, are used for that one request and aren’t kept. See Data retention.

Create a saved prompt

Send a POST request to /v1/prompts with a name and a prompt. The prompt is a JSON string of a chat completions request body.
The response is the saved prompt, with 201 Created:
The name and prompt must meet these limits: A request that breaks these limits returns 400, with each problem listed in metadata.errors. Creating a name your account already uses returns 409.

Run a saved prompt

Send a request to /v1/chat/completions with aai_prompt set to the prompt’s name.
The stored prompt replaces the request, so the model, messages and every other setting come from the prompt. The response is a normal chat completion.
Fields you send alongside aai_prompt are ignored, with these exceptions: prompt_replacements and transcript_id. Those apply to the stored prompt. model is optional when aai_prompt is set.
If your account has no prompt with that name, the request returns 404.

Fill in tags

Put {{ name }} placeholders in the messages of a saved prompt, then set prompt_replacements to an object of values to substitute:
  • A tag is {{, a space, the key, a space, then }}. Variants without the spaces, like {{team}}, aren’t replaced.
  • Every occurrence of a tag is replaced, in message text, in each part of a message with multiple content parts, and in prompt.
  • Tags are also replaced in the prompt’s fallbacks, one level deep. Tags in fallbacks nested inside a fallback are left as they are.
  • Replacement runs once. If a value contains a tag, that tag isn’t replaced in turn.
  • A tag with no matching key is left as it is.
prompt_replacements requires aai_prompt. Sending it without aai_prompt returns 400.

Use a transcript

The transcript tag is reserved. Put {{ transcript }} in a saved prompt and pass transcript_id when you run it, and LLM Gateway injects the transcript text as described in Inject a transcript by ID. Without a transcript_id, the tag is left as it is. Setting transcript in prompt_replacements returns 400.

Get, update and delete a saved prompt

Each of these takes the prompt’s name in the path.
A prompt that belongs to another account returns 404, the same as one that doesn’t exist.

Errors

See the API reference for the full request and response of each endpoint.