LLM Gateway Overview

Overview

AssemblyAI’s LLM Gateway is a unified interface that allows you to connect with multiple LLM providers including Claude, GPT, Gemini, and more. You can use the LLM Gateway to build sophisticated AI applications through a single API.

The LLM Gateway provides access to 20+ models across major AI providers with support for:

• Basic Chat Completions - Simple request/response interactions
• Streamed Responses - Stream output as it’s generated (OpenAI models)
• Multi-turn Conversations - Maintain context across multiple exchanges
• Structured Outputs - Constrain responses to a specific JSON schema
• Tool/Function Calling - Enable models to execute custom functions
• Agentic Workflows - Multi-step reasoning with automatic tool chaining
• Unified Interface - One API for Claude, GPT, Gemini, Qwen, Kimi, and more

Available models

By quality (LMArena Score)

By latency (per 10,000 tokens)

By provider

Anthropic Claude

OpenAI GPT

Google Gemini

Alibaba Cloud Qwen

Moonshot AI Kimi

Select a model

You can specify which model to use in your request by setting the model parameter. Here are examples showing how to use Claude 4.5 Sonnet:

1
import requests
2
3
headers = {
4
  "authorization": "<YOUR_API_KEY>"
5
}
6
7
response = requests.post(
8
    "https://llm-gateway.assemblyai.com/v1/chat/completions",
9
    headers = headers,
10
    json = {
11
        "model": "claude-sonnet-4-5-20250929",
12
        "messages": [
13
            {"role": "user", "content": "What is the capital of France?"}
14
        ],
15
        "max_tokens": 1000
16
    }
17
)
18
19
result = response.json()
20
print(result["choices"][0]["message"]["content"])

Simply change the model parameter to use any of the available models listed in the Available models section above.

Next steps

• Basic Chat Completions - Learn how to send simple messages and receive responses
• Multi-turn Conversations - Maintain context across multiple exchanges
• Structured Outputs - Constrain model responses to follow a specific JSON schema
• Tool Calling - Enable models to execute custom functions
• Agentic Workflows - Build multi-step reasoning applications