Tool Calling | AssemblyAI

Overview

Tool calling allows you to define functions that the model can call to access external data or perform actions. This enables the model to interact with databases, APIs, and other external systems.

Defining tools

Define functions the model can call to access external data or functionality:

1 import requests
2 
3 headers = {
4   "authorization": "<YOUR_API_KEY>"
5 }
6 
7 tools = [
8     {
9         "type": "function",
10         "function": {
11             "name": "search_employees",
12             "description": "Search employee database by name, department, or other criteria",
13             "parameters": {
14                 "type": "object",
15                 "properties": {
16                     "first_name": {
17                         "type": "string",
18                         "description": "Employee's first name"
19                     },
20                     "department": {
21                         "type": "string",
22                         "description": "Department name"
23                     }
24                 },
25                 "required": []
26             }
27         }
28     }
29 ]
30 
31 response = requests.post(
32     "https://llm-gateway.assemblyai.com/v1/chat/completions",
33     headers = headers,
34     json={
35         "model": "claude-sonnet-4-5-20250929",
36         "messages": [{"role": "user", "content": "Find employees in Engineering"}],
37         "tools": tools
38     }
39 )
40 
41 result = response.json()
42 print(result["choices"][0]["message"]["content"])

Handling tool calls

Execute tools and send results back to continue the conversation:

1 import requests
2 import json
3 
4 headers = {
5   "authorization": "<YOUR_API_KEY>"
6 }
7 
8 choice = response.json()["choices"][0]
9 
10 if choice.get("tool_calls"):
11     tool_call = choice["tool_calls"][0]
12 
13     # Add function call to history
14     conversation_history.append({
15         "type": "function_call",
16         "tool_call_id": tool_call["id"],
17         "name": tool_call["function"]["name"],
18         "arguments": tool_call["function"]["arguments"]
19     })
20 
21     # Execute your function
22     result = execute_function(
23         tool_call["function"]["name"],
24         json.loads(tool_call["function"]["arguments"])
25     )
26 
27     # Add function output to history
28     conversation_history.append({
29         "type": "function_call_output",
30         "tool_call_id": tool_call["id"],
31         "name": tool_call["function"]["name"],
32         "output": json.dumps({"result": result})
33     })
34 
35     # Continue conversation with the result
36     response = requests.post(
37         "https://llm-gateway.assemblyai.com/v1/chat/completions",
38         headers = headers,
39         json = {
40             "model": "claude-sonnet-4-5-20250929",
41             "messages": conversation_history,
42             "tools": tools
43         }
44     )
45 
46 result = response.json()
47 print(result["choices"][0]["message"]["content"])

Tool call flow

The complete tool calling process:

Send user message with tools array defining available functions
Model responds with tool_calls if it needs to use a tool
Execute the tool in your application code
Add function_call and function_call_output to conversation history
Send updated history back to the API
Model incorporates results and either calls more tools or provides final answer

Message types

When using tools, structure your conversation history with these message types to track the complete interaction flow:

user - Messages from the user
assistant - Messages from the AI model
system - System instructions or context
function_call - Records a tool being called by the model
function_call_output - Records the result of executing a tool

API reference

Request

The LLM Gateway accepts POST requests to https://llm-gateway.assemblyai.com/v1/chat/completions with the following parameters:

$ curl -X POST \
>   "https://llm-gateway.assemblyai.com/v1/chat/completions" \
>   -H "Authorization: YOUR_API_KEY" \
>   -H "Content-Type: application/json" \
>   -d '{
>     "model": "claude-sonnet-4-5-20250929",
>     "messages": [
>       {
>         "role": "user",
>         "content": "Find employees in Engineering"
>       }
>     ],
>     "tools": [
>       {
>         "type": "function",
>         "function": {
>           "name": "search_employees",
>           "description": "Search employee database by name, department, or other criteria",
>           "parameters": {
>             "type": "object",
>             "properties": {
>               "department": {
>                 "type": "string",
>                 "description": "Department name"
>               }
>             }
>           }
>         }
>       }
>     ],
>     "max_tokens": 1000
>   }'

Request parameters

Key	Type	Required?	Description
`model`	string	Yes	The model to use for completion. See Available models section for supported values.
`messages`	array	Yes	An array of message objects representing the conversation history.
`tools`	array	Yes	An array of tool definitions that the model can call.
`tool_choice`	string or object	No	Controls which tools the model can call. Options: `"none"`, `"auto"`, or an object specifying a specific function.
`max_tokens`	number	No	The maximum number of tokens to generate. Range: [1, context_length).
`temperature`	number	No	Controls randomness in the output. Higher values make output more random. Range: [0, 2].

Message object

Key	Type	Required?	Description
`role`	string	Yes	The role of the message sender. Valid values: `"user"`, `"assistant"`, `"system"`, or `"tool"`.
`content`	string or array	Yes	The message content. Can be a string or an array of content parts for the `"user"` role.
`name`	string	No	An optional name for the message sender. For non-OpenAI models, this will be prepended as `{name}: {content}`.
`tool_call_id`	string	No*	Required when `role` is `"tool"`. The ID of the tool call this message is responding to.

Tool object

Key	Type	Required?	Description
`type`	string	Yes	The type of tool. Currently only `"function"` is supported.
`function`	object	Yes	The function definition.
`function.name`	string	Yes	The name of the function.
`function.description`	string	No	A description of what the function does.
`function.parameters`	object	Yes	A JSON Schema object describing the function parameters.

Tool choice object

The tool_choice parameter can be:

"none" - The model will not call any tools
"auto" - The model can choose whether to call tools
An object with type: "function" and a function.name to force calling a specific function

Response

The API returns a JSON response. When the model wants to call a tool:

1 {
2   "request_id": "abc123",
3   "choices": [
4     {
5       "message": {
6         "role": "assistant",
7         "content": null
8       },
9       "finish_reason": "tool_calls",
10       "tool_calls": [
11         {
12           "id": "call_123",
13           "type": "function",
14           "function": {
15             "name": "search_employees",
16             "arguments": "{\"department\": \"Engineering\"}"
17           }
18         }
19       ]
20     }
21   ],
22   "request": {
23     "model": "claude-sonnet-4-5-20250929",
24     "max_tokens": 1000
25   },
26   "usage": {
27     "prompt_tokens": 120,
28     "completion_tokens": 25,
29     "total_tokens": 145
30   }
31 }

Response fields

Key	Type	Description
`request_id`	string	A unique identifier for the request.
`choices`	array	An array of completion choices. Typically contains one choice.
`choices[i].message`	object	The message object containing the model’s response.
`choices[i].message.role`	string	The role of the message, typically `"assistant"`.
`choices[i].message.content`	string	The text content of the model’s response. May be null when calling tools.
`choices[i].finish_reason`	string	The reason the model stopped generating. Common values: `"stop"`, `"length"`, `"tool_calls"`.
`choices[i].tool_calls`	array	Present when the model wants to call tools. Contains function call objects.
`request`	object	Echo of the request parameters (excluding `messages`).
`usage`	object	Token usage statistics for the request.
`usage.prompt_tokens`	number	Number of tokens in the prompt.
`usage.completion_tokens`	number	Number of tokens in the completion.
`usage.total_tokens`	number	Total tokens used (prompt + completion).

Tool call object

When the model wants to call a tool, the response includes a tool_calls array:

Key	Type	Description
`tool_calls[i].id`	string	A unique identifier for the tool call.
`tool_calls[i].type`	string	The type of tool call, always `"function"`.
`tool_calls[i].function`	object	The function call details.
`tool_calls[i].function.name`	string	The name of the function to call.
`tool_calls[i].function.arguments`	string	A JSON string containing the function arguments.

Error response

If an error occurs, the API returns an error response:

1 {
2   "error": {
3     "code": 400,
4     "message": "Invalid request: missing required field 'model'",
5     "metadata": {}
6   }
7 }

Key	Type	Description
`error`	object	Container for error information.
`error.code`	number	HTTP status code for the error.
`error.message`	string	A human-readable description of the error.
`error.metadata`	object	Optional additional error context.

Common error codes

Code	Description
400	Bad Request - Invalid request parameters
401	Unauthorized - Invalid or missing API key
403	Forbidden - Insufficient permissions
404	Not Found - Invalid endpoint or model
429	Too Many Requests - Rate limit exceeded
500	Internal Server Error - Server-side error