Basic Chat Completions

Overview

Basic chat completions allow you to send a message and receive a response from the model. This is the simplest way to interact with the LLM Gateway.

Getting started

Send a message and receive a response:

1
import requests
2
3
headers = {
4
  "authorization": "<YOUR_API_KEY>"
5
}
6
7
response = requests.post(
8
    "https://llm-gateway.assemblyai.com/v1/chat/completions",
9
    headers = headers,
10
    json = {
11
        "model": "claude-sonnet-4-6",
12
        "messages": [
13
            {"role": "user", "content": "What is the capital of France?"}
14
        ],
15
        "max_tokens": 1000
16
    }
17
)
18
19
result = response.json()
20
print(result["choices"][0]["message"]["content"])

Streamed responses

You can stream responses from OpenAI models by setting stream to true. This returns partial responses as server-sent events (SSE), allowing you to display output as it’s generated.

1
import requests
2
3
headers = {
4
  "authorization": "<YOUR_API_KEY>"
5
}
6
7
response = requests.post(
8
    "https://llm-gateway.assemblyai.com/v1/chat/completions",
9
    headers=headers,
10
    json={
11
        "model": "gpt-4.1",
12
        "messages": [
13
            {"role": "user", "content": "What is the capital of France?"}
14
        ],
15
        "stream": True,
16
        "max_tokens": 1000
17
    },
18
    stream=True
19
)
20
21
for line in response.iter_lines():
22
    if line:
23
        print(line.decode("utf-8"))

API reference

Request

The LLM Gateway accepts POST requests to https://llm-gateway.assemblyai.com/v1/chat/completions with the following parameters:

$
curl -X POST \
>
  "https://llm-gateway.assemblyai.com/v1/chat/completions" \
>
  -H "Authorization: YOUR_API_KEY" \
>
  -H "Content-Type: application/json" \
>
  -d '{
>
    "model": "claude-sonnet-4-6",
>
    "messages": [
>
      {
>
        "role": "user",
>
        "content": "What is the capital of France?"
>
      }
>
    ],
>
    "max_tokens": 1000
>
  }'

Request parameters

Message object

Content part object

Response

The API returns a JSON response with the model’s completion:

1
{
2
  "request_id": "abc123",
3
  "choices": [
4
    {
5
      "message": {
6
        "role": "assistant",
7
        "content": "The capital of France is Paris."
8
      },
9
      "finish_reason": "stop"
10
    }
11
  ],
12
  "request": {
13
    "model": "claude-sonnet-4-6",
14
    "max_tokens": 1000
15
  },
16
  "usage": {
17
    "input_tokens": 15,
18
    "output_tokens": 8,
19
    "total_tokens": 23
20
  }
21
}

Response fields

Inject a transcript by ID

Pass transcript_id at the top level of the request to inject a transcript’s text into the prompt. The API replaces the first occurrence of the literal tag {{ transcript }} in the first message containing it with the transcript’s text field, then runs the completion.

$
curl -X POST "https://llm-gateway.assemblyai.com/v1/chat/completions" \
>
  -H "Authorization: YOUR_API_KEY" \
>
  -H "Content-Type: application/json" \
>
  -d '{
>
    "model": "gemini-2.5-flash-lite",
>
    "messages": [
>
      {"role": "user", "content": "hi there"},
>
      {"role": "assistant", "content": "Hi! How can I help?"},
>
      {"role": "user", "content": "Here is a transcript: {{ transcript }}. Return the text verbatim."}
>
    ],
>
    "transcript_id": "065a71ac-dc3e-4e38-9374-e54c0bea564f"
>
  }'

Error response

If an error occurs, the API returns an error response:

1
{
2
  "error": {
3
    "code": 400,
4
    "message": "Invalid request: missing required field 'model'",
5
    "metadata": {}
6
  }
7
}

Common error codes