Skip to main content

​
Overview

Post-processing steps let you apply automatic fixes to model responses after generation. You can specify an ordered list of steps in the post_processing_steps parameter on any chat completions request. Steps run server-side on all LLM Gateway models in both US and EU regions. Currently, JSON repair (json-repair) is the only supported step type.

​
JSON repair

JSON repair corrects common JSON errors — such as trailing commas, unescaped characters, and missing quotes — that LLMs occasionally produce. This is especially useful when using structured outputs or tool calling, where invalid JSON would otherwise require client-side retry logic. If the response content cannot be repaired, the request returns an error rather than passing through broken JSON.

​
Getting started

Add post_processing_steps to any chat completions request: 
import requests
import json

headers = {
    "authorization": "<YOUR_API_KEY>",
    "content-type": "application/json"
}

response = requests.post(
    "https://llm-gateway.assemblyai.com/v1/chat/completions",
    headers=headers,
    json={
        "model": "gemini-2.5-flash-lite",
        "messages": [
            {
                "role": "user",
                "content": "Extract the user name and age from: John is 30 years old. Return as JSON."
            }
        ],
        "post_processing_steps": [{"type": "json-repair"}]
    }
)

result = response.json()
parsed = json.loads(result["choices"][0]["message"]["content"])
print(parsed)

​
What JSON repair fixes

The JSON repair step corrects the most common JSON errors produced by LLMs:
Error typeExample (broken)After repair
Trailing comma{"name": "John",}{"name": "John"}
Unescaped characters{"note": "say "hi""}{"note": "say \"hi\""}
Missing closing bracket{"name": "John"{"name": "John"}
Single-quoted strings{'name': 'John'}{"name": "John"}
The step applies to both message content and tool call arguments in the response.
If the JSON cannot be repaired, the request returns an HTTP 500 error. The raw malformed response is never passed through.

​
Combining with structured outputs

post_processing_steps works independently of response_format. You can use both together for maximum reliability: 
{
  "model": "gemini-2.5-flash-lite",
  "messages": [{"role": "user", "content": "..."}],
  "response_format": {
    "type": "json_schema",
    "json_schema": { "name": "result", "schema": { ... }, "strict": true }
  },
  "post_processing_steps": [{"type": "json-repair"}]
}

​
API reference

​
post_processing_steps parameter

KeyTypeRequired?Description
post_processing_stepsarrayNoAn ordered list of post-processing steps to apply to the response. Omit to disable post-processing.
post_processing_steps[i].typestringYesThe step type. Currently "json-repair" is supported.

​
Supported step types

TypeApplies toModelsRegions
json-repairMessage content and tool call argumentsAll LLM Gateway modelsUS and EU

​
Full request example

curl -X POST "https://llm-gateway.assemblyai.com/v1/chat/completions" \
    -H "Content-Type: application/json" \
    -H "Authorization: <YOUR_API_KEY>" \
    -d '{
      "model": "gemini-2.5-flash-lite",
      "messages": [
        {
          "role": "user",
          "content": "Extract the user name and age from: John is 30 years old. Return as JSON."
        }
      ],
      "post_processing_steps": [{"type": "json-repair"}]
    }'