Documentation Index Fetch the complete documentation index at: https://assemblyai.com/docs/llms.txt
Use this file to discover all available pages before exploring further.
Overview Tool calling allows you to define functions that the model can call to access external data or perform actions. This enables the model to interact with databases, APIs, and other external systems.
Tool call objects follow an OpenAI-compatible structure nested under choices[i].message.tool_calls, so the same parser works across Claude, OpenAI, and Gemini. finish_reason is passed through provider-native (tool_use / end_turn for Claude, tool_calls / stop for OpenAI), so handle both families when branching on it.
Define functions the model can call to access external data or functionality:
import requests headers = { "authorization" : "<YOUR_API_KEY>" } tools = [ { "type" : "function" , "function" : { "name" : "search_employees" , "description" : "Search employee database by name, department, or other criteria" , "parameters" : { "type" : "object" , "properties" : { "first_name" : { "type" : "string" , "description" : "Employee's first name" }, "department" : { "type" : "string" , "description" : "Department name" } }, "required" : [] } } } ] response = requests.post( "https://llm-gateway.assemblyai.com/v1/chat/completions" , headers = headers, json = { "model" : "claude-sonnet-4-6" , "messages" : [{ "role" : "user" , "content" : "Find employees in Engineering" }], "tools" : tools } ) result = response.json() print (result[ "choices" ][ 0 ][ "message" ][ "content" ])
Execute tools and send results back to continue the conversation:
import requests import json headers = { "authorization" : "<YOUR_API_KEY>" } choice = response.json()[ "choices" ][ 0 ] if choice.get( "message" , {}).get( "tool_calls" ): tool_call = choice[ "message" ][ "tool_calls" ][ 0 ] # Add assistant message with tool calls to history conversation_history.append({ "role" : "assistant" , "content" : None , "tool_calls" : [tool_call] }) # Execute your function result = execute_function( tool_call[ "function" ][ "name" ], json.loads(tool_call[ "function" ][ "arguments" ]) ) # Add tool result to history conversation_history.append({ "role" : "tool" , "tool_call_id" : tool_call[ "id" ], "content" : json.dumps({ "result" : result}) }) # Continue conversation with the result response = requests.post( "https://llm-gateway.assemblyai.com/v1/chat/completions" , headers = headers, json = { "model" : "claude-sonnet-4-6" , "messages" : conversation_history, "tools" : tools } ) result = response.json() print (result[ "choices" ][ 0 ][ "message" ][ "content" ])
The complete tool calling process:
Send user message with tools array defining available functions
Model responds with tool_calls if it needs to use a tool
Execute the tool in your application code
Add the assistant message (with tool_calls) and a tool message (with the result) to conversation history
Send updated history back to the API
Model incorporates results and either calls more tools or provides final answer
Message types When using tools, structure your conversation history with these message types to track the complete interaction flow:
user - Messages from the userassistant - Messages from the AI modelsystem - System instructions or contexttool - Results from executing a tool call
API reference Request The LLM Gateway accepts POST requests to https://llm-gateway.assemblyai.com/v1/chat/completions with the following parameters:
curl -X POST \ "https://llm-gateway.assemblyai.com/v1/chat/completions" \ -H "Authorization: YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "claude-sonnet-4-6", "messages": [ { "role": "user", "content": "Find employees in Engineering" } ], "tools": [ { "type": "function", "function": { "name": "search_employees", "description": "Search employee database by name, department, or other criteria", "parameters": { "type": "object", "properties": { "department": { "type": "string", "description": "Department name" } } } } } ], "max_tokens": 1000 }'
Request parameters Key Type Required? Description modelstring Yes The model to use for completion. See Available models section for supported values. messagesarray Yes An array of message objects representing the conversation history. toolsarray Yes An array of tool definitions that the model can call. tool_choicestring or object No Controls which tools the model can call. Options: "none", "auto", or an object specifying a specific function. max_tokensnumber No The maximum number of tokens to generate. Default: 1000. Range: [1, context_length). temperaturenumber No Controls randomness in the output. Higher values make output more random. Range: [0, 2].
Message object Key Type Required? Description rolestring Yes The role of the message sender. Valid values: "user", "assistant", "system", or "tool". contentstring or array Yes The message content. Can be a string or an array of content parts for the "user" role. namestring No An optional name for the message sender. For non-OpenAI models, this will be prepended as {name}: {content}. tool_call_idstring No* Required when role is "tool". The ID of the tool call this message is responding to.
Key Type Required? Description typestring Yes The type of tool. Currently only "function" is supported. functionobject Yes The function definition. function.namestring Yes The name of the function. function.descriptionstring No A description of what the function does. function.parametersobject Yes A JSON Schema object describing the function parameters.
The tool_choice parameter can be:
"none" - The model will not call any tools"auto" - The model can choose whether to call toolsAn object with type: "function" and a function.name to force calling a specific function
Response The API returns a JSON response. When the model wants to call a tool:
{ "request_id" : "abc123" , "choices" : [ { "index" : 0 , "finish_reason" : "tool_use" , "message" : { "role" : "assistant" , "content" : "I'll search the engineering department." } }, { "index" : 1 , "finish_reason" : "tool_use" , "message" : { "role" : "assistant" , "content" : "" , "tool_calls" : [ { "id" : "call_123" , "type" : "function" , "function" : { "name" : "search_employees" , "arguments" : "{ \" department \" : \" Engineering \" }" } } ] } } ], "request" : { "model" : "claude-sonnet-4-6" , "max_tokens" : 1000 }, "usage" : { "input_tokens" : 120 , "output_tokens" : 25 , "total_tokens" : 145 } }
Response fields Key Type Description request_idstring A unique identifier for the request. choicesarray An array of completion choices. Typically contains one choice. choices[i].messageobject The message object containing the model’s response. choices[i].message.rolestring The role of the message, typically "assistant". choices[i].message.contentstring The text content of the model’s response. May be null when calling tools. choices[i].finish_reasonstring The reason the model stopped generating. Common values: "stop", "length", "tool_calls" (OpenAI models), "tool_use", "end_turn" (Claude models). LLM Gateway passes the provider-native value through; client code should accept either family. choices[i].message.tool_callsarray Present on assistant message choices when the model calls a tool. Each tool call appears in its own choice with the calling content nested under message. requestobject Echo of the request parameters (excluding messages). usageobject Token usage statistics for the request. usage.input_tokensnumber Number of tokens in the prompt. usage.output_tokensnumber Number of tokens in the completion. usage.total_tokensnumber Total tokens used (prompt + completion).
When the model wants to call a tool, the response includes a tool_calls array:
Key Type Description tool_calls[i].idstring A unique identifier for the tool call. tool_calls[i].typestring The type of tool call, always "function". tool_calls[i].functionobject The function call details. tool_calls[i].function.namestring The name of the function to call. tool_calls[i].function.argumentsstring A JSON string containing the function arguments.
Models occasionally return malformed JSON in tool_calls[i].function.arguments. Add post_processing_steps: [{"type": "json-repair"}] to your request to automatically repair common JSON errors. See Post-processing . Error response If an error occurs, the API returns an error response:
{ "error" : { "code" : 400 , "message" : "Invalid request: missing required field 'model'" , "metadata" : {} } }
Key Type Description errorobject Container for error information. error.codenumber HTTP status code for the error. error.messagestring A human-readable description of the error. error.metadataobject Optional additional error context.
Common error codes Code Description 400 Bad Request - Invalid request parameters 401 Unauthorized - Invalid or missing API key 403 Forbidden - Insufficient permissions 404 Not Found - Invalid endpoint or model 429 Too Many Requests - Rate limit exceeded 500 Internal Server Error - Server-side error
LLMs occasionally return malformed JSON in tool call arguments. Add post_processing_steps to your request to automatically repair these before they reach your application:
{ "post_processing_steps" : [{ "type" : "json-repair" }] }
See Post-processing for details. 
