Post-processing

Overview

Post-processing steps let you apply automatic fixes to model responses after generation. You can specify an ordered list of steps in the post_processing_steps parameter on any chat completions request. Steps run server-side on all LLM Gateway models in both US and EU regions.

Currently, JSON repair (json-repair) is the only supported step type.

JSON repair

JSON repair corrects common JSON errors — such as trailing commas, unescaped characters, and missing quotes — that LLMs occasionally produce. This is especially useful when using structured outputs or tool calling, where invalid JSON would otherwise require client-side retry logic.

If the response content cannot be repaired, the request returns an error rather than passing through broken JSON.

Getting started

Add post_processing_steps to any chat completions request:

1
import requests
2
import json
3
4
headers = {
5
    "authorization": "<YOUR_API_KEY>",
6
    "content-type": "application/json"
7
}
8
9
response = requests.post(
10
    "https://llm-gateway.assemblyai.com/v1/chat/completions",
11
    headers=headers,
12
    json={
13
        "model": "gemini-2.5-flash-lite",
14
        "messages": [
15
            {
16
                "role": "user",
17
                "content": "Extract the user name and age from: John is 30 years old. Return as JSON."
18
            }
19
        ],
20
        "post_processing_steps": [{"type": "json-repair"}]
21
    }
22
)
23
24
result = response.json()
25
parsed = json.loads(result["choices"][0]["message"]["content"])
26
print(parsed)

What JSON repair fixes

The JSON repair step corrects the most common JSON errors produced by LLMs:

The step applies to both message content and tool call arguments in the response.

Combining with structured outputs

post_processing_steps works independently of response_format. You can use both together for maximum reliability:

1
{
2
  "model": "gemini-2.5-flash-lite",
3
  "messages": [{"role": "user", "content": "..."}],
4
  "response_format": {
5
    "type": "json_schema",
6
    "json_schema": { "name": "result", "schema": { ... }, "strict": true }
7
  },
8
  "post_processing_steps": [{"type": "json-repair"}]
9
}

API reference

post_processing_steps parameter

Supported step types

Full request example

$
curl -X POST "https://llm-gateway.assemblyai.com/v1/chat/completions" \
>
    -H "Content-Type: application/json" \
>
    -H "Authorization: <YOUR_API_KEY>" \
>
    -d '{
>
      "model": "gemini-2.5-flash-lite",
>
      "messages": [
>
        {
>
          "role": "user",
>
          "content": "Extract the user name and age from: John is 30 years old. Return as JSON."
>
        }
>
      ],
>
      "post_processing_steps": [{"type": "json-repair"}]
>
    }'