New AssemblyAI-TTS-1: Text-to-speech for voice agents. Learn more
Insights & Use Cases

How to summarize meetings with AI in Python: summaries, action items, and which LLM to use

Two ways to build an AI meeting scribe in Python: built-in Summarization and Action Items in the transcription request, or custom prompts through LLM Gateway with the model picked by meeting length.

Abstract green half-sphere illustration

Written by

Ryan O'Connor

Published on

20 October 2026

To build an AI meeting scribe in Python, transcribe the recording with speaker labels using AssemblyAI's speech-to-text API on Universal-3.5 Pro, then get the summary and action items one of two ways. Path A turns on Summarization and Action Items from Speech Understanding in the same transcription request. Path B sends the transcript to a model you pick through LLM Gateway with your own prompt and output format.

Both paths depend on the same first step, and that step decides more than the model does. A summarization prompt can't recover a name the transcriber got wrong, and it can't attribute a decision to the right person if the speaker boundaries are in the wrong place. Every hallucinated action item in a meeting summary traces back to a word the model never had.

This guide builds both paths end to end, then answers the question most tutorials skip: which LLM to use for a 30-second clip versus an hour-long meeting. The answer changes with length, and we have the benchmark data to show it.

What you need

An AssemblyAI API key (the free tier covers up to 185 hours of pre-recorded audio), Python 3.9 or later, and one package. Every example here uses plain HTTP, so you can read exactly what goes over the wire.

pip install requests
export ASSEMBLYAI_API_KEY="<YOUR_API_KEY>"

Step 1: Transcribe the meeting with speaker labels

Meeting audio is the hard case for speech-to-text: overlapping speakers, cross-talk, laptop mics, someone dialing in from a car. Name the model explicitly instead of relying on a default. You want to know what produced your transcript, and you want it pinned when you compare output across runs.

Universal-3.5 Pro is the flagship async model at $0.21/hr. It handles 18 languages with native code-switching, which matters when a participant switches languages mid-sentence. The speech_models parameter takes a list, so universal-2 (99 languages, $0.15/hr) acts as the fallback for anything outside those 18. Turn on speaker_labels for diarization (+$0.02/hr). A summary that doesn't know who said what can't tell you who owns the follow-up.

These helpers upload a local file, submit the request, and poll until it's done. Everything later in this guide reuses them.

import os
import time
from typing import Optional

import requests

API_KEY = os.environ.get("ASSEMBLYAI_API_KEY", "<YOUR_API_KEY>")
HEADERS = {"authorization": API_KEY}
STT_BASE = "https://api.assemblyai.com"
LLM_URL = "https://llm-gateway.assemblyai.com/v1/chat/completions"


def upload(path: str) -> str:
    """Upload a local recording and return a URL the API can read."""
    with open(path, "rb") as f:
        r = requests.post(f"{STT_BASE}/v2/upload", headers=HEADERS, data=f)
    r.raise_for_status()
    return r.json()["upload_url"]


def transcribe(audio_url: str, **options) -> dict:
    """Submit a transcription request and poll until it finishes."""
    payload = {
        "audio_url": audio_url,
        "speech_models": ["universal-3-5-pro", "universal-2"],
        "speaker_labels": True,
        **options,
    }
    r = requests.post(f"{STT_BASE}/v2/transcript", json=payload, headers=HEADERS)
    r.raise_for_status()
    transcript_id = r.json()["id"]

    while True:
        r = requests.get(f"{STT_BASE}/v2/transcript/{transcript_id}", headers=HEADERS)
        r.raise_for_status()
        t = r.json()
        if t["status"] == "completed":
            return t
        if t["status"] == "error":
            raise RuntimeError(f"Transcription failed: {t['error']}")
        time.sleep(3)

Keep the transcript ID. The transcript is the expensive artifact, and you can run as many summaries over it later as you like without transcribing again.

Which path should you use?

There are two ways to summarize text. Extractive summarization pulls the most important sentences straight from the source; abstractive summarization writes new sentences that capture the meaning. Both paths below are abstractive, because that's what reads like meeting notes. The difference is who owns the prompt.

Path A: built-in Summarization + Action ItemsPath B: your prompt through LLM Gateway
Where it runsIn the transcription request (speech_understanding)A separate chat-completions call after transcription
OutputTopic summaries with headline and start/end timestamps; action items with a supporting quote and timestampWhatever you specify: Markdown sections, JSON, a CRM note
Model choiceManaged for youAny of 50+ models; you pick per job
Prompt maintenanceNoneYours to version and test
Best forShipping a meeting scribe fast, standard notes and to-dosCustom formats, owners and due dates, writing into other systems

Plenty of teams run both: Path A for the default notes every user sees, Path B for the one format a specific workflow needs.

Path A: summaries and action items in the same request

Add a speech_understanding object to the transcription request and you get the transcript, topic-based summaries, and a list of action items back in one response. No second pipeline, no chunking, no re-aligning LLM output to timestamps. We covered the design in the Summarization and Action Items launch; here's the code.

transcript = transcribe(
    upload("weekly-sync.mp3"),
    speech_understanding={
        "request": {
            "summarization": {"summary_type": "bullets"},
            "action_items": {},
        }
    },
)

su = transcript["speech_understanding"]["response"]

summary = su["summarization"]
if summary["status"] == "success":
    for topic in summary["summary"]:
        start, end = topic["start"] / 1000, topic["end"] / 1000
        print(f"[{start:.0f}s-{end:.0f}s] {topic['headline']}")
        # "bullets" topics carry a bullets list; "paragraph" topics carry text
        for line in topic.get("bullets") or [topic.get("text", "")]:
            print(f"  - {line}")

items = su.get("action_items", {})
if items.get("status") == "success":
    for item in items["items"]:
        at = item["timestamp"] / 1000
        print(f"- {item['action_item']} (at {at:.0f}s: \"{item['quote']}\")")

A few things to know about the response:

  • Summaries are per topic, not one block. Each entry in summary covers one topic and carries a headline plus start and end times in milliseconds, so you can link every section back to the recording or render a table of contents with no post-processing.
  • summary_type controls the shape. It's required (there's no default), so set it to bullets or paragraph: bullets returns a short bullets list per topic; paragraph returns longer prose in text.
  • Action items come with evidence. Each item has the action_item, the quote it came from, and a timestamp. That quote is what lets a user trust the item without replaying the call.
  • effort trades cost for detail. Both features accept effort: "low" (default) or "medium". Use medium for high-stakes meetings, multilingual audio, or recordings of about 1.5 hours and longer.
  • Check status first. Each feature reports its own status, so a summary failure doesn't take down the transcript.

Summarization costs +$0.02/hr at low effort and +$0.07/hr at medium, on top of the speech-to-text rate. Both Summarization and Action Items are in open beta. If you're still sending the old top-level summarization, summary_model, or auto_chapters parameters, those are deprecated and being turned off; moving to speech_understanding.request.summarization is a config change.

Put real names on the speakers

"Speaker B owns the migration" doesn't get acted on. "Priya owns the migration" does. Speaker attribution is the thing users complain about most in AI meeting notes, and Speaker Identification fixes it in the same request: give it the participant names (from the calendar invite, say) and short descriptions, and it maps the diarized labels to real names. It costs +$0.02/hr at the default low effort.

request = {
    "summarization": {"summary_type": "paragraph"},
    "action_items": {},
    "speaker_identification": {
        "speaker_type": "name",
        "speakers": [
            {"name": "Priya", "description": "Engineering lead who owns the database migration"},
            {"name": "Marcus", "description": "Product manager who runs the weekly sync"},
        ],
    },
}

transcript = transcribe(upload("weekly-sync.mp3"), speech_understanding={"request": request})

for u in transcript["utterances"]:
    print(f"{u['speaker']}: {u['text']}")

Use speaker_type "role" instead when you know roles but not names, such as Interviewer and Candidate. Speaker Identification can only use names that are present in the context you give it or in the conversation, so feed it the attendee list whenever you have one. You can combine it with Summarization and Action Items in one request, as above, but the summaries and action items don't use the identified names, and speakers that don't match a description keep their letter labels (Speaker A, Speaker B).

Get summaries and action items in one request

Start free with Universal-3.5 Pro. Speech Understanding runs in the same call, so your first meeting scribe is one request away.

Sign up free

Path B: custom summaries with LLM Gateway

When the built-in format isn't the format your product needs, write the prompt yourself. LLM Gateway gives you one endpoint and one API key for 50+ models from OpenAI, Anthropic, Google and others, plus open models AssemblyAI hosts directly, at provider rates with 0% markup. LLM Gateway requires a paid account, so you'll need one for Path B.

Write the prompt as an output spec

Treat the prompt as a specification, not a request. Vague prompts produce vague summaries, and the failure mode is confident invention: a model asked to "summarize the meeting" fills gaps instead of reporting them. Four things belong in every meeting prompt: a role and scope, named sections in a fixed order, an instruction to write "Not discussed" instead of guessing, and a rule to keep product names and acronyms as spoken.

Send the transcript by ID

You don't need to paste the transcript into your request. Pass the transcript_id and put the literal tag {{ transcript }} in your message; the Gateway swaps in the transcript text server-side. The tag must be exactly {{ transcript }}, with the spaces, and only the first one in the first message that contains it is replaced.

# Models where global routing is available. Omit model_region for open models
# such as glm-5.3-flash or qwen3.5-4b-32k-fast, and for in-region processing.
GLOBAL_ROUTING_OK = {"claude-opus-5-5", "gemini-4-argon", "gpt-6-luna"}


def ask_gateway(model: str, messages: list, transcript_id: Optional[str] = None, **extra) -> dict:
    """Send a chat completion request to LLM Gateway and return the JSON response."""
    payload = {"model": model, "messages": messages, "max_tokens": 4000, **extra}
    if transcript_id:
        payload["transcript_id"] = transcript_id
    if model in GLOBAL_ROUTING_OK:
        payload["model_region"] = "global"

    for attempt in range(5):
        r = requests.post(LLM_URL, json=payload, headers=HEADERS, timeout=300)
        if r.status_code == 429:  # rate limited: back off and retry
            time.sleep(2 ** attempt)
            continue
        r.raise_for_status()
        return r.json()
    raise RuntimeError("LLM Gateway kept returning 429. Lower your concurrency.")


SUMMARY_PROMPT = """You are summarizing an internal team meeting.
Return Markdown with these sections, in this order:
## Summary (3 to 5 sentences)
## Decisions
## Action items (task, due date if one was stated)
## Open questions
If the transcript has nothing for a section, write "Not discussed." Do not guess.
Keep product names and acronyms exactly as spoken.

Transcript:
{{ transcript }}"""

result = ask_gateway(
    "claude-opus-5-5",
    [{"role": "user", "content": SUMMARY_PROMPT}],
    transcript_id=transcript["id"],
)
print(result["choices"][0]["message"]["content"])

Two details in that helper are worth copying. First, model_region: "global" opts into global routing, which is what the listed token prices assume; in-region (US or EU) processing is 10% higher for Claude, Gemini, and GPT models. Only send it for models that support it: check each model's available_regions from GET https://llm-gateway.assemblyai.com/v1/models before relying on global routing (gemini-4-argon in particular). Leave it off for open models such as glm-5.3-flash and qwen3.5-4b-32k-fast, which carry one price. Second, the helper sets max_tokens to 4000, because the Gateway's default of 1000 can cut off a long meeting summary. The {{ transcript }} tag injects plain text without speaker labels, so for owner attribution pass formatted utterances instead, as in the structured action-items example below.

Change the prompt and you change the product. The same call produces a project status report, a sales call recap, or a support QA review; only the sections change.

Action items as JSON, with owners and timestamps

To push action items into Jira, Linear, or a CRM, you want data, not prose. This version sends speaker-labeled lines with timestamps (so the model can attribute and cite) and constrains the response to a JSON schema. The json-repair post-processing step fixes common formatting slips, like a trailing comma, before the response reaches your parser.

import json


def format_utterances(utterances: list) -> str:
    """Flatten utterances into '[mm:ss] Speaker: text' lines."""
    lines = []
    for u in utterances:
        minutes, seconds = divmod(int(u["start"] // 1000), 60)
        lines.append(f"[{minutes:02d}:{seconds:02d}] {u['speaker']}: {u['text']}")
    return "\n".join(lines)


ACTION_ITEM_SCHEMA = {
    "type": "object",
    "properties": {
        "action_items": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "owner": {"type": "string"},
                    "task": {"type": "string"},
                    "due": {"type": "string"},
                    "timestamp": {"type": "string"},
                },
                "required": ["owner", "task", "due", "timestamp"],
                "additionalProperties": False,
            },
        }
    },
    "required": ["action_items"],
    "additionalProperties": False,
}

prompt = (
    "Extract every action item from this meeting. Use the speaker as the owner. "
    "Copy the [mm:ss] timestamp of the line where the commitment was made. "
    'If no due date was stated, use "unspecified".\n\n'
    + format_utterances(transcript["utterances"])
)

result = ask_gateway(
    "claude-opus-5-5",
    [{"role": "user", "content": prompt}],
    response_format={
        "type": "json_schema",
        "json_schema": {"name": "action_items", "schema": ACTION_ITEM_SCHEMA, "strict": True},
    },
    post_processing_steps=[{"type": "json-repair"}],
)

items = json.loads(result["choices"][0]["message"]["content"])["action_items"]
for item in items:
    print(f"{item['timestamp']} {item['owner']}: {item['task']} (due: {item['due']})")

If you ran Speaker Identification on the transcript, the speaker field already holds real names, so the owner field does too.

Why diarization quality decides summary quality

Speaker diarization segments audio by speaker. When it drifts, an action item lands on the wrong person, and the summary is worse than useless because it's confidently wrong.

That's why the number to watch is cpWER (concatenated minimum-permutation word error rate), not DER. DER measures whether speaker boundaries line up in time. cpWER measures what your summarizer actually consumes: whether the right words ended up attributed to the right speaker. Lower is better. On the AssemblyAI benchmarks page:

ModelAverage cpWER
AssemblyAI Universal-3.5 Pro30.17
Microsoft Azure30.35
ElevenLabs Scribe v235.26
Deepgram Nova-337.92

That gap compounds across a 60-minute call with six participants. Context helps too. Metaview, which builds AI notes for recruiting conversations, put it this way:

"Since moving to AssemblyAI, we've seen a meaningful improvement in the confidence tail of our production transcripts....What stands out is not just the model quality, but the way [they] let us bring real meeting context into transcription, from calendar titles to organizations, domains, and participant names, so recruiting conversations come through with the nuance our customers depend on."

— Shahriar Tajbakhsh, Co-founder and CTO, Metaview

Pick the LLM by meeting length

"Which model should I use to summarize meeting transcripts?" is the most common question in this space, and general-purpose indexes don't answer it. They rank models on reasoning and coding over clean typed text, not on a messy hour of cross-talk.

Voice Standard Bench does. It scores every model on LLM Gateway on real speech-to-text output, with the filler words and false starts left in, across four tasks and four transcript lengths: about 30 seconds, 2 minutes, 15 minutes, and 60 minutes. Two judge models, Claude Opus 5.5 and GPT-5.6 Sol, grade each summary on faithfulness (no claims the transcript doesn't support), coverage (the topics, decisions, and action items that matter), and conciseness. Speed and cost are reported beside quality.

Here are the summarization picks from the October 2026 run. Quality is 0 to 100, averaged across transcript lengths, and the price and latency picks must land within 8 points of the top score:

PickModel (API ID)Summarization qualityPrice per 1M tokens (in / out)
Top qualityOpus 5.5 (claude-opus-5-5)98$4.00 / $20.00
Lowest price within 8 pointsGLM-5.3 Flash (glm-5.3-flash)91$0.15 / $0.50
Lowest latency within 8 pointsGemini 4 Argon (gemini-4-argon)93$2.00 / $10.00

The more useful finding is about length. Small, fast models win on short transcripts and fall off at meeting length. Qwen3.5 4B Fast is the fastest model in the run at every length up to 15 minutes, and its summarization quality shows the curve:

Transcript lengthQwen3.5 4B Fast summarization qualityWhere to start
30 seconds (voicemail, clip)76A fast open model is fine; latency matters more than the last few points
2 minutes (short call)69Fast models still competitive; compare against the budget pick
15 minutes (standup, support call)60Move to the latency or budget pick from the table above
60 minutes (meeting)55Start with the top-quality pick

Top picks score 95 to 99 at the same lengths. Price follows the same pattern: across the run, the top scorer costs 15 to 77 times more per hour of audio than a model within two quality points of it, but on hour-long transcripts that premium narrows to about 4×. For meetings, the bigger model is the cheaper mistake. An incomplete summary of a long meeting is expensive to fix by hand.

A starting rule in code:

def pick_model_for_length(transcript: dict) -> str:
    """Starting points from Voice Standard Bench. Re-test on your own meetings."""
    minutes = transcript["audio_duration"] / 60
    if minutes <= 2:
        return "qwen3.5-4b-32k-fast"  # fastest model in the run on short clips
    if minutes <= 20:
        return "gemini-4-argon"  # lowest-latency summarization pick within 8 points of the top
    return "claude-opus-5-5"  # top summarization quality in the current run


model = pick_model_for_length(transcript)
result = ask_gateway(model, [{"role": "user", "content": SUMMARY_PROMPT}], transcript["id"])

If you need the budget pick, glm-5.3-flash runs on an opt-in provider, so enable opt-in providers in your dashboard's data controls first. If your prompts are already tuned on claude-sonnet-4-6, it still works through the Gateway; run it through the same comparison before you assume it's the right default. Bench picks move as new models arrive, and the full catalog with IDs, supported parameters, and prices is on the LLM Gateway models page.

Now the important part. Our transcripts aren't yours. Your speakers have different accents, your domain has its own vocabulary, your prompt asks for a different shape, and your cost limit is your own. Use the bench to pick two or three models to try, then test them on your own meetings. Swapping models is a one-word change to the model field. You can also run the open-source Voice Standard Bench code against your own transcripts, with the same tasks, rubric, and graders.

Compare three models on your own meeting

Upload a recording and run the same summary prompt across the bench picks side by side in Playground compare mode. No code needed.

Try playground

Handle very long meetings

A one-hour meeting fits comfortably in the context window of the long-context picks above (Opus 5.5 and Gemini 4 Argon both accept up to about a million tokens), so for most meetings the right move is to skip chunking and send the whole transcript to one strong model. Chunking adds seams, and seams are where attribution breaks.

Map-reduce still earns its keep in two cases: multi-hour recordings, and pipelines where you want a fast open model to do most of the token work. If you go that route, use short windows. The bench says fast models hold up at about 2 minutes and fall off by 15, so map over roughly 2-minute windows, split on utterance boundaries so you never cut a speaker turn in half, and give the reduce step to a stronger model.

def chunk_utterances(utterances: list, window_ms: int = 120_000) -> list:
    """Group whole utterances into windows of about window_ms, never splitting a turn."""
    chunks, current, window_start = [], [], None
    for u in utterances:
        if window_start is None:
            window_start = u["start"]
        if current and u["end"] - window_start > window_ms:
            chunks.append(current)
            current, window_start = [], u["start"]
        current.append(u)
    if current:
        chunks.append(current)
    return chunks


MAP_PROMPT = (
    "List the decisions, action items (with owner) and open questions in this "
    "meeting excerpt. Write 'none' for any list that is empty.\n\n"
)
REDUCE_PROMPT = (
    "These notes cover consecutive parts of one meeting. Merge them into a single "
    "summary with Decisions, Action items and Open questions. Remove duplicates "
    "and keep every owner.\n\n"
)


def summarize_long_meeting(
    transcript: dict,
    map_model: str = "qwen3.5-4b-32k-fast",
    reduce_model: str = "claude-opus-5-5",
) -> str:
    partials = []
    for chunk in chunk_utterances(transcript["utterances"]):
        r = ask_gateway(map_model, [{"role": "user", "content": MAP_PROMPT + format_utterances(chunk)}])
        partials.append(r["choices"][0]["message"]["content"])

    merged = "\n\n---\n\n".join(partials)
    r = ask_gateway(reduce_model, [{"role": "user", "content": REDUCE_PROMPT + merged}])
    return r["choices"][0]["message"]["content"]

Check summary quality before you ship

You can't ship summaries you haven't measured. Word-overlap metrics such as ROUGE are fast and close to useless for abstractive summaries, because a great summary can use entirely different words from the transcript. What works is an LLM grading the output against the source, which is the same approach Voice Standard Bench uses at scale.

Send the transcript and the generated summary to a judge model and ask for the bench's three scores plus a list of unsupported claims. Use a judge from a different model family than the one that wrote the summary, since LLM judges tend to favor output from their own family.

JUDGE_PROMPT = """You are grading a meeting summary against the transcript it came from.
Score each dimension from 1 to 5:
- faithfulness: every claim in the summary appears in the transcript
- coverage: the summary captures the topics, decisions and action items that matter
- conciseness: no small talk, filler or repeated points
List every claim in the summary that the transcript does not support.

Transcript:
{{ transcript }}

Summary to grade:
"""

JUDGE_SCHEMA = {
    "type": "object",
    "properties": {
        "faithfulness": {"type": "integer"},
        "coverage": {"type": "integer"},
        "conciseness": {"type": "integer"},
        "unsupported_claims": {"type": "array", "items": {"type": "string"}},
    },
    "required": ["faithfulness", "coverage", "conciseness", "unsupported_claims"],
    "additionalProperties": False,
}


def judge_summary(transcript_id: str, summary: str, judge_model: str = "gpt-6.1-sol") -> dict:
    """Grade a summary with a model from a different family than the one that wrote it."""
    r = ask_gateway(
        judge_model,
        [{"role": "user", "content": JUDGE_PROMPT + summary}],
        transcript_id=transcript_id,
        response_format={
            "type": "json_schema",
            "json_schema": {"name": "summary_grade", "schema": JUDGE_SCHEMA, "strict": True},
        },
        post_processing_steps=[{"type": "json-repair"}],
    )
    return json.loads(r["choices"][0]["message"]["content"])


grade = judge_summary(transcript["id"], result["choices"][0]["message"]["content"])
if grade["faithfulness"] < 4 or grade["unsupported_claims"]:
    print("Review before sending:", grade["unsupported_claims"])

Run it on a sample of production summaries every time you change a prompt or a model. It costs one extra call per summary, which is cheap insurance against sending a recap that invents a quarterly target nobody mentioned.

Summarize a backlog in parallel

Transcription and summarization both take real time, so don't block a web request on them. Submit transcriptions in the background, store the IDs, and run the summaries concurrently. Because the Gateway pulls each transcript by ID, a nightly job over yesterday's calls is a few lines:

import asyncio


async def summarize_many(transcript_ids: list, model: str, max_concurrency: int = 5) -> dict:
    """Summarize many stored transcripts in parallel, capped at max_concurrency."""
    sem = asyncio.Semaphore(max_concurrency)

    async def one(tid: str):
        async with sem:
            r = await asyncio.to_thread(
                ask_gateway, model, [{"role": "user", "content": SUMMARY_PROMPT}], tid
            )
            return tid, r["choices"][0]["message"]["content"]

    pairs = await asyncio.gather(*(one(tid) for tid in transcript_ids))
    return dict(pairs)


# The transcript IDs you stored when you submitted yesterday's recordings
yesterdays_transcript_ids = ["<TRANSCRIPT_ID_1>", "<TRANSCRIPT_ID_2>"]

summaries = asyncio.run(summarize_many(yesterdays_transcript_ids, "gemini-4-argon"))

Keep concurrency modest and let the helper back off on 429 responses; the Gateway rate-limits per model. Three more things before this goes to production:

  • Check transcription status, not just HTTP status. Transcription fails asynchronously, so a failed job returns a status of error rather than raising. The helper raises on it; don't swallow that exception.
  • Add a fallback model. With no fallbacks set, a failed request is retried once after 500 ms. The Gateway also supports model fallbacks, so a provider outage downgrades to your runner-up instead of failing the job. The bench's second pick for your length is a sensible fallback.
  • Store transcripts separately from summaries. If a prompt change next month means you want different summaries, you shouldn't have to re-transcribe anything.

Keep meeting recordings private

Meeting audio is some of the most sensitive data a company holds: compensation discussions, legal strategy, customer names, unannounced roadmaps. Two controls matter when you put it through an LLM.

Zero data retention. LLM Gateway applies zero data retention to inputs and outputs by default, so your summaries don't sit in a log.

Regional processing. For data that must stay in the EU, point both transcription and the Gateway at the EU endpoints. The EU Gateway endpoint serves Claude and Gemini models only, so swap the Qwen, GLM, and GPT defaults used elsewhere in this guide (such as qwen3.5-4b-32k-fast for the map step and gpt-6.1-sol as the judge) for a Claude or Gemini model ID. Opus 5.5 and Gemini 4 Argon are both on the bench's summarization shortlist. Don't opt into global routing.

# EU data residency: transcription and LLM Gateway both stay in the EU
STT_BASE = "https://api.eu.assemblyai.com"
LLM_URL = "https://llm-gateway.eu.assemblyai.com/v1/chat/completions"

# Global routing sends requests to providers' global endpoints, so turn it off here.
# The EU endpoint serves Claude and Gemini models only.
GLOBAL_ROUTING_OK = set()

Listed LLM Gateway token prices are for global routing; in-region (US or EU) processing is 10% higher for Claude, Gemini, and GPT models. Build that into your per-meeting cost model up front.

No code? Try it in the Playground

You can see both paths before writing a line of Python. Upload a meeting recording in the AssemblyAI Playground, get the transcript with speaker labels, and turn on summaries. Then send the transcript to a few LLM Gateway models with your own prompt and compare the answers side by side.

Practical notes on quality

Audio input. Most bad summaries are bad recordings. Individual microphones beat a single room mic by a wide margin, because diarization has real channel separation to work with. Record locally where you can rather than relying on a compressed conference-bridge stream.

Prompt discipline. Version prompts alongside your code and diff summary outputs when you change them. An edit that improves action-item extraction can quietly degrade decision capture, and you won't notice without a before-and-after. The judge above is how you notice.

Meeting structure. A verbal recap of decisions before the meeting ends is the single most useful habit. It hands the model an unambiguous statement of every decision to anchor on.

Wrapping up

The pipeline is short. Transcribe with speech_models ["universal-3-5-pro", "universal-2"] and speaker_labels, then either turn on Summarization and Action Items in the same request or send the transcript ID to LLM Gateway with a prompt that specifies the output. Everything else is tuning.

The shift worth noticing is that the model choice isn't a one-time decision anymore. The right model for a 30-second voicemail is the wrong one for an hour-long planning meeting, and the best pick for either will change as new models land. Teams that treat the model as a per-job setting, chosen by length and re-tested on their own transcripts, get better notes for less money than teams that hard-code one model everywhere. If you're building AI notetaker infrastructure, the same transcript can also generate next-meeting agendas, and if your stack is JavaScript, here's the same pipeline in Node.js.

Which LLM for your voice data? Start with our pick, then prove it on your data

Voice Standard Bench gives you two or three models to try first. Take those recommendations and test them on your own meeting transcripts with LLM Gateway, using the same API key as your speech-to-text.

Test models in LLM Gateway

Frequently asked questions

How do I create an AI meeting scribe that generates summaries and action items?

Transcribe the meeting with AssemblyAI's async API on Universal-3.5 Pro with speaker_labels turned on, and add speech_understanding.request.summarization and action_items to the same request. The response returns the transcript, topic-based summaries with headlines and timestamps, and action items with supporting quotes and timestamps. Add Speaker Identification with the attendee list so the transcript's speaker labels become real names instead of "Speaker B." The built-in summaries and action items don't use those names, so for named owners send the speaker-labeled transcript to LLM Gateway.

Is there an API to create bullet-point summaries from transcripts?

Yes. AssemblyAI's Summarization feature returns bullet-point summaries per topic when you set summary_type to "bullets," or paragraph summaries if you set it to "paragraph." summary_type is required; there's no default. If you need a different bullet format, send the transcript to any LLM Gateway model with a prompt that specifies the bullets you want.

Which LLM is best for summarizing meeting transcripts?

It depends on meeting length. In Voice Standard Bench, small fast models such as Qwen3.5 4B Fast do well on short clips but drop from 76 quality at 30 seconds to 55 at 60 minutes, while the top summarization pick, Opus 5.5, averages 98. For hour-long meetings start with a top pick, consider Gemini 4 Argon for lower latency or GLM-5.3 Flash for lower cost, and test two or three candidates on your own transcripts before you choose.

How do I extract action items from a meeting transcript with timestamps?

Add action_items to speech_understanding.request in the transcription request, and each returned item includes the action, the supporting quote, and a timestamp in milliseconds. For custom fields such as owner and due date, send speaker-labeled, timestamped lines to LLM Gateway with a JSON schema in response_format. Either way, check the status field before reading results.

How accurate is automated summarization of audio content?

Summary accuracy is bounded by transcript accuracy and speaker attribution, because a model can't summarize words it never received or credit a decision to the right person if diarization put it on the wrong speaker. That's why diarization accuracy measured by cpWER matters more for meeting summaries than raw word error rate. On the summarization step itself, Voice Standard Bench shows quality holding up for most models on short and medium transcripts, with a wider spread as transcripts get longer.

How do I summarize meetings longer than an hour?

For meetings up to a few hours, send the full transcript to a long-context model such as Opus 5.5 or Gemini 4 Argon instead of chunking, since every chunk boundary is a chance to lose attribution. For longer recordings, use map-reduce: split on utterance boundaries into roughly 2-minute windows for a fast open model, summarize each, then merge the partial summaries with a stronger model. With built-in Summarization, set effort to "medium" for audio of about 1.5 hours or longer.