Skip to main content
Most agents should create a stored agent instead. Storing your configuration server-side keeps secrets off the client, runs HTTP tools for you, and lets you deploy the same agent across the API, a browser, and Twilio by agent_id. This page covers the inline alternative: configuring an agent at connect-time without storing it, useful for fully dynamic or one-off agents. The field reference below applies to both: the same fields exist on a stored agent’s input/output objects.
session.update is the configuration surface for an inline (non-stored) agent. Everything that shapes how the agent sounds and behaves lives here: its personality, its greeting, the tools it can call, how it interprets user speech, and what voice it speaks in. Send a session.update as your first WebSocket message, and any time after, to update most fields. Some fields can only be set in the first update; see Mutability after session.ready below.
To use a stored agent instead, send { "agent_id": "<id>" } as the only field in your first session.update; it’s mutually exclusive with the inline fields below. See Deploy your agent.
Here’s a full configuration showing every available field: 
{
  "type": "session.update",
  "session": {
    "system_prompt": "You are a friendly support agent. Keep responses under 2 sentences.",
    "greeting": "Hi! How can I help you today?",
    "tools": [],
    "input": {
      "format": { "encoding": "audio/pcm" },
      "keyterms": ["AssemblyAI", "Universal"],
      "turn_detection": {
        "vad_threshold": 0.5,
        "min_silence": 1000,
        "max_silence": 3000,
        "interrupt_response": true
      }
    },
    "output": {
      "voice": "ivy",
      "format": { "encoding": "audio/pcm" },
      "volume": 100
    }
  }
}
Every field is optional. Include only what you want to set or change. Jump to any section below for details.

​
Mutability after session.ready

The first session.update you send before session.ready initializes the session. After session.ready, only a subset of fields can be changed. Changing one of the immutable fields raises a session.error with code immutable_field and the rejected change is ignored.
FieldMutable after session.ready?
session.system_promptYes. Send a new prompt at any time to change the agent’s behavior on the next turn.
session.input.turn_detectionYes. Adjust VAD thresholds, silence windows, and barge-in on the fly.
session.input.keytermsYes. Replace the keyterms list at any time. The new list takes effect on the next user utterance.
session.output.volumeYes. Adjust playback volume on the fly.
session.greetingNo. Raises immutable_field. The greeting is spoken once at session start.
session.output.voiceNo. Raises immutable_field. The voice is bound to the TTS connection at session start.
session.output.formatNo. Raises immutable_field. The output audio encoding is fixed for the session.
Other fields (session.tools, session.input.format) are also accepted in subsequent session.update messages and don’t raise immutable_field.

​
Fields

The session object carries the same fields as a stored agent. Each has a dedicated guide with details and examples; the cURL there maps one-to-one onto the session.update shape shown above.
  • system_prompt — the agent’s personality and behavior. Updatable mid-session. See the Prompting guide.
  • greeting — the exact words spoken on connect, sent straight to TTS. Immutable after session.ready. See Greeting.
  • output.voice — the TTS voice, e.g. "ivy". Immutable after session.ready. See Voices.
  • input.format / output.format — audio encoding (default audio/pcm at 24 kHz). output.format is immutable after session.ready. See Audio encoding.
  • output.volume — playback volume, 0–100. Updatable mid-session. See Volume.
  • input.keyterms — up to 100 transcription-bias terms. Updatable mid-session. See Key terms.
  • input.turn_detection — VAD tuning and barge-in. Updatable mid-session. See Turn detection and interruptions.
  • tools — actions the agent can call. See Add tools.