Voice AI PlatformLLM Gateway

Structured Outputs

Overview

Structured outputs allow you to constrain the model’s response to follow a specific JSON schema. This ensures the model returns data in a predictable format that can be reliably parsed and processed by your application.

Structured outputs are supported by OpenAI (GPT-4.1, GPT-5.x) and Gemini models. Claude and gpt-oss models do not currently support structured outputs.

Getting started

To use structured outputs, include the response_format parameter in your request with a json_schema type:

1import requests
2
3headers = {
4    "authorization": "<YOUR_API_KEY>",
5    "content-type": "application/json"
6}
7
8response = requests.post(
9    "https://llm-gateway.assemblyai.com/v1/chat/completions",
10    headers=headers,
11    json={
12        "model": "gemini-2.5-flash-lite",
13        "messages": [
14            {
15                "role": "system",
16                "content": "You are a helpful math tutor. Guide the user through the solution step by step."
17            },
18            {
19                "role": "user",
20                "content": "how can I solve 8x + 7 = -23"
21            }
22        ],
23        "response_format": {
24            "type": "json_schema",
25            "json_schema": {
26                "name": "math_reasoning",
27                "schema": {
28                    "type": "object",
29                    "properties": {
30                        "steps": {
31                            "type": "array",
32                            "items": {
33                                "type": "object",
34                                "properties": {
35                                    "explanation": {"type": "string"},
36                                    "output": {"type": "string"}
37                                },
38                                "required": ["explanation", "output"],
39                                "additionalProperties": False
40                            }
41                        },
42                        "final_answer": {"type": "string"}
43                    },
44                    "required": ["steps", "final_answer"],
45                    "additionalProperties": False
46                },
47                "strict": True
48            }
49        }
50    }
51)
52
53result = response.json()
54print(result["choices"][0]["message"]["content"])

Example response

When using structured outputs, the model’s response will be a JSON string that conforms to your schema:

1{
2  "request_id": "abc123",
3  "choices": [
4    {
5      "message": {
6        "role": "assistant",
7        "content": "{\"steps\":[{\"explanation\":\"Start with the equation 8x + 7 = -23\",\"output\":\"8x + 7 = -23\"},{\"explanation\":\"Subtract 7 from both sides to isolate the term with x\",\"output\":\"8x = -30\"},{\"explanation\":\"Divide both sides by 8 to solve for x\",\"output\":\"x = -30/8 = -15/4 = -3.75\"}],\"final_answer\":\"x = -3.75\"}"
8      },
9      "finish_reason": "stop"
10    }
11  ],
12  "usage": {
13    "input_tokens": 85,
14    "output_tokens": 120,
15    "total_tokens": 205
16  }
17}

You can parse the content as JSON in your application:

1import json
2
3content = result["choices"][0]["message"]["content"]
4parsed = json.loads(content)
5
6for step in parsed["steps"]:
7    print(f"{step['explanation']}: {step['output']}")
8
9print(f"Final answer: {parsed['final_answer']}")

Supported models

Structured outputs are supported by the following model families:

ProviderSupported
OpenAI (GPT-4.1, GPT-5.x)Yes
GeminiYes
gpt-ossNo
ClaudeNo

API reference

Request parameters

The response_format parameter controls how the model formats its response:

KeyTypeRequired?Description
response_formatobjectNoSpecifies the format of the model’s response.
response_format.typestringYesThe type of response format. Use "json_schema" for structured outputs.
response_format.json_schemaobjectYesThe JSON schema configuration object.

JSON schema object

KeyTypeRequired?Description
json_schema.namestringYesA name for the schema. Used for identification purposes.
json_schema.schemaobjectYesA valid JSON Schema object that defines the structure of the expected response.
json_schema.strictbooleanNoWhen true, the model will strictly adhere to the schema. Recommended for reliable parsing.

Schema definition

The schema object follows the JSON Schema specification. Common properties include:

PropertyTypeDescription
typestringThe data type: "object", "array", "string", "number", "boolean".
propertiesobjectFor objects, defines the properties and their schemas.
itemsobjectFor arrays, defines the schema for array items.
requiredarrayList of required property names.
additionalPropertiesbooleanWhen false, prevents additional properties not defined in the schema.

Best practices

When using structured outputs, keep these recommendations in mind:

Set strict: true to ensure the model’s response strictly adheres to your schema. This is especially important when your application depends on specific fields being present.

Use additionalProperties: false at each level of your schema to prevent the model from adding unexpected fields to the response.

Keep your schemas focused and specific. Complex schemas with many nested levels may increase latency and token usage.

Include clear descriptions in your system or user messages to help the model understand what data to extract or generate for each field.

Error handling

If the model cannot generate a valid response that matches your schema, you may receive an error or a response that doesn’t fully conform to the schema. Always validate the parsed JSON against your expected structure:

1import json
2
3try:
4    content = result["choices"][0]["message"]["content"]
5    parsed = json.loads(content)
6
7    # Validate required fields exist
8    if "steps" not in parsed or "final_answer" not in parsed:
9        raise ValueError("Missing required fields in response")
10
11except json.JSONDecodeError as e:
12    print(f"Failed to parse response as JSON: {e}")
13except KeyError as e:
14    print(f"Unexpected response structure: {e}")