Choose a model

Overview

LLM Gateway gives you access to dozens of models, and the best one depends on your task, how long your audio is, and whether you care most about quality, speed, or cost. The models endpoint answers this for you. Alongside each model’s pricing and supported parameters, it returns a performance block with benchmark results for voice post-processing tasks. The numbers come from Voice Standard Bench, AssemblyAI’s open source benchmark. It runs every model behind LLM Gateway on real call transcripts and measures quality, latency, and cost.

Field names and values can change as the benchmark grows, so read it from the endpoint instead of copying numbers into your code.

How the numbers work

Tasks

Each model is tested on four voice post-processing tasks. Your model’s score for each task is in task_quality.

Task	Key in `task_quality`	What the model does	How it’s scored
Summarization	`summarization`	Summarizes a call transcript.	An LLM judge scores faithfulness, coverage, and conciseness against the source transcript.
Information extraction	`information_extraction`	Pulls participants, action items, decisions, and entities from a transcript as JSON.	An LLM judge scores groundedness and completeness against the source transcript.
Text correction	`text_correction`	Cleans up disfluencies and errors in a transcript without changing its meaning.	An LLM judge scores faithfulness and improvement.
Dictation	`dictation`	Applies spoken commands such as “comma”, “all caps”, and “new paragraph”.	Scored mechanically against an answer key, with no LLM judge.

Quality scores run from 1 to 100, where higher is better. The overall quality_score is the equal-weight average of the tasks that have data.

Audio length buckets

Every row in performance.data is for one audio length, in the bucket field: 30s, 120s, 900s, or 3600s. Pick the bucket closest to the transcripts you process. Scores generally drop as transcripts get longer, so a model that is excellent on 30-second calls can be a different choice for hour-long recordings.

Fields

Field	Meaning
`quality_score`	Combined quality, 1 to 100.
`task_quality`	Quality per task, 1 to 100.
`cost_per_hour_usd`	Estimated cost to process one hour of audio. Use this to compare cost across models, buckets, and tasks. `task_cost_per_hour_usd` gives the same estimate per task.
`total_cost_usd`, `cost_per_quality`	Cost of the benchmark samples behind the score. `cost_per_quality` is cost per call per quality point, in cents, where lower is more efficient.
`latency`	End-to-end request latency in seconds, with `p50`, `p95`, `mean`, `min`, `max`, and sample count `n`.
`ttft`	Time to first token in seconds from a separate streaming run, with the same statistics as `latency`.
`tokens_per_second`	Output speed.
`reasoning_effort`	The reasoning setting the row was measured with. See Reasoning effort rows.

Reading the numbers carefully

A 0 means no data, not a bad score. Dictation isn’t measured on the 900s and 3600s buckets, so its task_quality is 0 there. A ttft of 0 means it wasn’t measured. Skip zeros instead of ranking on them.
Small gaps are noise. A one- or two-point difference between models doesn’t tell you much. Prefer the cheaper or faster model when scores are close.
Cross-check outliers. If a single task score is far above the model’s quality_score or its other tasks, it may come from very few usable samples. Check latency.n and the other tasks before trusting it.
Look at p95, not just p50. A model with a fast median can still have a slow tail, which matters for real-time use.
Costs use global pricing. They don’t include the regional surcharge shown on the Models page. cost_per_hour_usd is an estimate based on average tokens per hour of audio, so fast or slow talkers shift your real cost.
The benchmark uses fixed prompts. Your prompts, your audio, and your domain will differ. Treat the numbers as a way to build a shortlist, then test the shortlist on your own data.

Reasoning effort rows

Most models have one row per bucket with reasoning_effort set to default. That means the benchmark didn’t send the reasoning_effort parameter. A few models have extra rows measured with a specific value, such as none, low, or high. Each row is a separate configuration with its own quality, latency, and cost, so a lower setting can be much faster and cheaper at a small cost in quality. To use a non-default row, send that value in the reasoning_effort parameter of your chat completions request. Only models that list reasoning_effort in supported_parameters accept it.

Pick a model

Work through these filters in order. They’re the same ones the coding agent prompt below applies.

Match the bucket. Choose the bucket closest to your typical audio length. If your audio falls between buckets, use the longer one.
Rank by the task you run. Use task_quality for that task. Use quality_score only if you run several tasks with the same model.
Filter to what you can use. Drop models where retirement_date isn’t 0, models whose available_regions don’t include your region (see Cloud endpoints and data residency), and models whose supported_parameters lack what you need, such as response_format for structured outputs or tools for tool calling.
Apply your constraint. For lowest cost, sort by cost_per_hour_usd. For real-time responses, sort by latency.p50 and check latency.p95 (or ttft if you stream). For best quality, take the top score.
Break ties on cost. Among models within a couple of quality points of the top, choose the cheapest or fastest.
Add a fallback. Pick the runner-up as a fallback so requests keep working if your primary model fails.

Example from the data

This snapshot is from October 15, 2026. For each bucket it shows the top-scoring model and the cheapest model within 3 points of it, excluding models with a retirement date. Quality is the combined quality_score. The endpoint is the source of truth, so query it for current results.

Audio length	Goal	Model	Quality	Cost per hour	p50 latency	Regions
`30s`	Top score	`gpt-6-luna` (`high`)	96	$0.0054	2.5 s	US, global
`30s`	Cheapest within 3 points	`gpt-6-luna` (`none`)	94	$0.0028	0.9 s	US, global
`120s`	Top score	`gemini-4-argon`	94	$0.073	2.6 s	US, EU, global
`120s`	Cheapest within 3 points	`gemma-4-31b` (`none`)	91	$0.0036	1.7 s	US
`900s`	Top score	`claude-opus-5`	89	$0.32	14.9 s	US, global
`900s`	Cheapest within 3 points	`gpt-6-luna`	87	$0.0048	15.5 s	US, global
`3600s`	Top score	`gpt-6-astra`	88	$0.35	24.1 s	US
`3600s`	Cheapest within 3 points	`gpt-6-luna`	85	$0.0039	19.0 s	US, global

Models with no setting in parentheses were measured at default reasoning effort. Per-task winners differ from these overall winners, so rank on task_quality for your task before you decide.

Query the models endpoint

The endpoint is public, so you don’t need an API key. This example lists the five best models for summarization on 2-minute audio, skipping retiring models and models with no summarization data:

curl -s https://llm-gateway.assemblyai.com/v1/models | jq -r '
  [ .data[] | select(.retirement_date == 0) | . as $m
    | .performance.data[]
    | select(.bucket == "120s" and .task_quality.summarization > 0)
    | { model: $m.id, reasoning_effort, summarization: .task_quality.summarization,
        cost_per_hour_usd, p50_latency_s: .latency.p50 } ]
  | sort_by(-.summarization) | .[:5][]
  | [.model, .reasoning_effort, .summarization, .cost_per_hour_usd, .p50_latency_s] | @tsv'

Each model in the response has this shape. Fields not used for model selection are omitted here:

{
  "id": "gpt-6-luna",
  "available_regions": ["us", "global"],
  "retirement_date": 0,
  "supported_parameters": ["max_tokens", "tools", "tool_choice", "stream", "response_format", "reasoning_effort"],
  "performance": {
    "status": "beta",
    "data": [
      {
        "model": "gpt-6-luna",
        "reasoning_effort": "default",
        "bucket": "30s",
        "quality_score": 95,
        "task_quality": { "dictation": 97, "information_extraction": 88, "summarization": 97, "text_correction": 96 },
        "cost_per_hour_usd": 0.00427,
        "latency": { "n": 100, "p50": 1.694, "p95": 5.055 },
        "ttft": { "n": 50, "p50": 0.992, "p95": 3.565 }
      }
    ]
  }
}

To use the EU endpoint, replace llm-gateway.assemblyai.com with llm-gateway.eu.assemblyai.com.

Choose a model with a coding agent

Describe what you’re building in plain language, and your coding agent can read the endpoint, work out which tasks, audio length, and constraints apply, and write the integration code for the model it picks. Paste this prompt into Claude Code, Cursor, or your agent of choice, and replace the bracketed text with your description. For example: “a meeting notetaker that summarizes hour-long recordings and lists action items, with data kept in the EU.”

Help me choose the best AssemblyAI LLM Gateway model for what I'm building, then wire it into my code.

What I'm building: [describe what you want to do with your audio or transcripts]

Work out my requirements from that description and from my codebase, instead of asking me for a list of settings. Infer:
- Which of the benchmark's tasks I need: summarization, information_extraction, text_correction, and/or dictation. A use case can need more than one, such as meeting notes (summarization and information_extraction) or cleaning up a transcript before display (text_correction).
- How long my audio is, and so which benchmark bucket to use: 30s, 120s, 900s, or 3600s.
- What I'm optimizing for: lowest cost, lowest latency, highest quality, or a balance. Real-time and interactive use points to latency. High-volume batch work points to cost. Clinical, legal, or customer-facing output points to quality.
- Which region I need. Mentions of EU, GDPR, or data residency point to eu.
- Which features I need, such as structured outputs (`response_format`), tool calling (`tools`), or streaming (`stream`).

State each of these as an assumption. Only ask me a question if the description leaves out something that would change the answer, and ask all such questions at once.

Then pick a model:
1. Fetch https://llm-gateway.assemblyai.com/v1/models. It is public and needs no API key.
2. Read https://www.assemblyai.com/docs/llm-gateway/choose-a-model.md to understand the `performance` fields before you rank anything.
3. Each model has `performance.data`, with one row per `bucket` and `reasoning_effort` value. Use the bucket closest to my audio length, rounding up to the longer bucket if it falls between two.
4. Rank on `task_quality[<task>]` for each task I need. If I need several tasks with one model, use their average, or `quality_score` if all four tasks have data.
5. Treat a `task_quality` value of 0 as "not measured", never as a poor score. Dictation has no data for 900s or 3600s.
6. Drop any model that has a non-zero `retirement_date`, whose `available_regions` doesn't include my region, or whose `supported_parameters` is missing something I need.
7. Compare cost with `cost_per_hour_usd`. Compare latency with `latency.p50` and `latency.p95`, and `ttft` if I stream.
8. Treat scores within about 2 points as ties and break them with my priority.
9. Each `reasoning_effort` row is a separate configuration. If the best row is not "default", send that value as the `reasoning_effort` request parameter.
10. Show me a shortlist of 3 models as a table with the score for my task, cost per hour, p50 and p95 latency, bucket, and reasoning effort. Explain the tradeoffs in one or two sentences each, and recommend one.

After I pick a model:
- Write the code to call POST /v1/chat/completions on the LLM Gateway with that model, reading my API key from the ASSEMBLYAI_API_KEY environment variable. Use https://llm-gateway.eu.assemblyai.com if I need EU data residency.
- Set the runner-up from the shortlist as a fallback using the `fallbacks` parameter. See https://www.assemblyai.com/docs/llm-gateway/fallback.md
- Put the model ID and reasoning effort in configuration, not hardcoded in the logic.

Do not copy benchmark scores into the code or comments. They change. Remind me that the numbers come from fixed benchmark prompts, and suggest I test the shortlist on a few of my own transcripts before going to production.

To keep your agent on current information in an ongoing project, see Build with AI agents for how to pin AssemblyAI docs in your project instructions.

Run Voice Standard Bench

The endpoint reflects one benchmark run on AssemblyAI’s own transcripts and prompts. Voice Standard Bench is open source (MIT), so you can see exactly how every number is produced and run it yourself. What it measures. For every model behind LLM Gateway, it runs the four tasks above on transcripts grouped by audio length. It records latency, token usage, cost, and quality, and rolls them up into the fields you see in the endpoint. What the repository gives you:

Published results. Open use_cases.html in results/canonical/ for a browsable report that groups the best models by use case and audio length, with no setup. summary.csv and summary.json hold every row.
The prompts. The task prompts, judge prompts, and scoring rubrics live in src/prompts. Read them to see what a score is measuring.
The harness. A Go CLI that benchmarks models through LLM Gateway. Use it to benchmark a specific shortlist of models, try a different reasoning_effort, or replay a run with a different judge without calling the models again.

When to run it yourself. Run it when the published numbers aren’t enough to decide, for example if your transcripts look very different from call audio, or you want to compare models on your own data. Otherwise the endpoint and the published results are the faster route.

The benchmark reads its transcripts from a normalized/ directory that isn’t included in the repository, and running it makes billable LLM Gateway calls. Read the README before you start. It covers the dataset layout, the flags, and how scores are calculated.

Models: every model with its ID, parameters, and pricing.
Model fallbacks: automatically retry on a backup model.
Cloud endpoints and data residency: which models are available in which regions.
Improve latency: reduce request overhead with keep-alive connections.

Getting started

Features

API reference

Core concepts

Voice AI Guides

Overview

How the numbers work

Tasks

Audio length buckets

Fields

Reading the numbers carefully

Reasoning effort rows

Pick a model

Example from the data

Query the models endpoint

Choose a model with a coding agent

Run Voice Standard Bench

​Overview

​How the numbers work

​Tasks

​Audio length buckets

​Fields

​Reading the numbers carefully

​Reasoning effort rows

​Pick a model

​Example from the data

​Query the models endpoint

​Choose a model with a coding agent

​Run Voice Standard Bench

​Related

Overview

How the numbers work

Tasks

Audio length buckets

Fields

Reading the numbers carefully

Reasoning effort rows

Pick a model

Example from the data

Query the models endpoint

Choose a model with a coding agent

Run Voice Standard Bench

Related