Events reference

Every message exchanged over the Voice Agent API WebSocket, grouped by direction. You’ll send session.update to configure, input.audio to stream mic audio, and tool.result to respond to tool calls. The server streams everything else back. For how these fit together in a typical session, see the Overview event flow.

Client → Server

input.audio

Stream PCM16 audio to the agent.

1
{
2
  "type": "input.audio",
3
  "audio": "<base64-encoded PCM16>"
4
}

See Audio format for the full format specification.

session.update

Configure the session. Send immediately on WebSocket connect (before session.ready). Can also be sent mid-conversation to update most fields. See Mutability after session.ready for which fields can change once the session is established.

1
{
2
  "type": "session.update",
3
  "session": {
4
    "system_prompt": "You are a concise assistant.",
5
    "greeting": "Hi! How can I help?",
6
    "input": {
7
      "format": { "encoding": "audio/pcm" },
8
      "turn_detection": { "vad_threshold": 0.5 }
9
    },
10
    "output": {
11
      "voice": "ivy",
12
      "format": { "encoding": "audio/pcm" }
13
    },
14
    "tools": [
15
      {
16
        "type": "function",
17
        "name": "get_weather",
18
        "description": "Get weather for a city",
19
        "parameters": {
20
          "type": "object",
21
          "properties": { "city": { "type": "string" } },
22
          "required": ["city"]
23
        }
24
      }
25
    ]
26
  }
27
}

All fields are optional. Include only what you want to set or change. After session.ready, only a subset of fields can be changed; changing greeting or session.output raises immutable_field.

session.resume

Reconnect to an existing session using the session_id from a previous session.ready. Preserves conversation context across dropped connections.

1
{
2
  "type": "session.resume",
3
  "session_id": "sess_abc123"
4
}

Example. Capture session_id from session.ready on the first connection, then send session.resume as the first message when reconnecting:

1
import json
2
import websockets
3
4
session_id: str | None = None
5
6
async def connect():
7
    global session_id
8
    async with websockets.connect(URL, additional_headers={"Authorization": f"Bearer {API_KEY}"}) as ws:
9
        # If we already have a session_id from a previous connection, resume it.
10
        if session_id:
11
            await ws.send(json.dumps({"type": "session.resume", "session_id": session_id}))
12
        else:
13
            await ws.send(json.dumps({"type": "session.update", "session": {...}}))
14
15
        async for raw in ws:
16
            event = json.loads(raw)
17
            if event["type"] == "session.ready":
18
                session_id = event["session_id"]  # save for next reconnect
19
            elif event["type"] == "session.error" and event["code"] in ("session_not_found", "session_forbidden"):
20
                session_id = None  # session expired - start fresh next time
21
            # ... handle other events
22
23
# On disconnect, call connect() again within 30 seconds to resume.

tool.result

Send a tool result back to the agent. Send this in the reply.done handler (not immediately in tool.call). See Tool calling.

1
{
2
  "type": "tool.result",
3
  "call_id": "call_abc123",
4
  "result": "{\"temp_c\": 22, \"description\": \"Sunny\"}"
5
}

Server → Client

session.ready

Session is established and ready to receive audio. Save session_id for reconnection. Start sending input.audio only after this event.

1
{
2
  "type": "session.ready",
3
  "session_id": "sess_abc123"
4
}

session.updated

Sent after session.update is applied successfully.

1
{ "type": "session.updated" }

input.speech.started

Turn detection determined the user has started speaking.

1
{ "type": "input.speech.started" }

input.speech.stopped

Turn detection determined the user has stopped speaking.

1
{ "type": "input.speech.stopped" }

transcript.user.delta

Partial transcript of what the user is saying, updating in real-time.

1
{
2
  "type": "transcript.user.delta",
3
  "text": "What's the weather in"
4
}

transcript.user

Final transcript of the user’s utterance.

1
{
2
  "type": "transcript.user",
3
  "text": "What's the weather in Tokyo?",
4
  "item_id": "item_abc123"
5
}

reply.started

Agent has begun generating a response.

1
{
2
  "type": "reply.started",
3
  "reply_id": "reply_abc123"
4
}

reply.audio

A chunk of the agent’s spoken response as base64 PCM16. Decode and play immediately.

1
{
2
  "type": "reply.audio",
3
  "data": "<base64-encoded PCM16>"
4
}

See Audio format for playback guidance.

transcript.agent

Full text of the agent’s response, sent after all audio for the response has been delivered. If the agent was interrupted, interrupted is true and text contains only what was actually spoken before the interruption.

1
{
2
  "type": "transcript.agent",
3
  "text": "It's currently 22°C and sunny in Tokyo.",
4
  "reply_id": "reply_abc123",
5
  "item_id": "item_abc123",
6
  "interrupted": false
7
}

reply.done

Agent has finished speaking. The optional status field indicates why the reply ended.

1
{ "type": "reply.done" }

1
{ "type": "reply.done", "status": "interrupted" }

tool.call

Agent wants to call a registered tool. arguments is a dict, ready to use directly as-is.

1
{
2
  "type": "tool.call",
3
  "call_id": "call_abc123",
4
  "name": "get_weather",
5
  "arguments": { "location": "Tokyo" }
6
}

See Tool calling for the full pattern.

session.error

Session or protocol error. The payload always includes type, timestamp, code, and message. Some errors (like session.update validation failures) also include a param field naming the offending field.

1
{
2
  "type": "session.error",
3
  "code": "invalid_format",
4
  "message": "Invalid message format",
5
  "timestamp": "2025-01-01T00:00:00Z"
6
}

Connection and handshake errors

Sent before or instead of session.ready. The WebSocket closes after these with the indicated close code.

Session resume errors

Sent when session.resume fails. The WebSocket closes after these.

Agent startup errors

Sent after the WebSocket is accepted but before session.ready.

Client message errors

Sent on the open socket when an inbound message is invalid. The session stays alive (except session_expired).

Live session errors

Interruptions

When the user speaks mid-response (barge-in), the server stops the agent and emits reply.done with status: "interrupted" and transcript.agent with interrupted: true. The decision is semantic. Back-channels like “uh-huh” don’t trigger an interruption. See Turn detection and interruptions for how the model decides, and Handling interruptions for the client-side flush pattern.