Summary

A common enhacement to minimize latency for http calls is to make use of a keep alive connection. This prevents a lot of the back and forth chatter between the server and your client and reduces latency on subsequent calls. The further your distance to the server, the more significant the improvement. Read more about this topic by searching “keep alive connections”.

How to do this

We can’t go over every possible library/language combination, so the high level recommendation is to lookup “set keep alive connection in insert lang/lib”. The some common cases:

Python requests use session = requests.Session()
The OpenAI sdk does this by default
JavaScript fetch set fetch(url, {keepalive: true})
Golang net/http does this by default

Python
JavaScript

import requests

headers = {
  "authorization": "<YOUR_API_KEY>"
}

# sets the keep alive connection
session = requests.Session()

response = session.post(
    "https://llm-gateway.assemblyai.com/v1/chat/completions",
    headers = headers,
    json = {
        "model": "claude-sonnet-4-6",
        "messages": [
            {"role": "user", "content": "What is the capital of France?"}
        ],
        "max_tokens": 1000
    }
)

result = response.json()
print(result["choices"][0]["message"]["content"])

const response = await fetch(
  "https://llm-gateway.assemblyai.com/v1/chat/completions",
  {
    keepalive: true, // sets the keep alive connection
    method: "POST",
    headers: {
      authorization: "<YOUR_API_KEY>",
      "content-type": "application/json",
    },
    body: JSON.stringify({
      model: "claude-sonnet-4-6",
      messages: [{ role: "user", content: "What is the capital of France?" }],
      max_tokens: 1000,
    }),
  }
);

const result = await response.json();
console.log(result.choices[0].message.content);

Getting started

Features

API reference

Core concepts

Voice AI Guides

Improve Latency

Summary

How to do this

​Summary

​How to do this

Summary

How to do this