Apply LLM Gateway to Audio Transcripts | AssemblyAI

Overview

A Large Language Model (LLM) is a machine learning model that uses natural language processing (NLP) to generate text. LLM Gateway is a unified API that provides access to 15+ models from Claude, GPT, and Gemini through a single interface. You can use LLM Gateway to analyze audio transcripts, for example to ask questions about a call, or to summarize a meeting.

By the end of this tutorial, you’ll be able to use LLM Gateway to summarize an audio file.

Here’s the full sample code for what you’ll build in this tutorial:

Python

JavaScript

C#

Ruby

PHP

1 import requests
2 import time
3 
4 # Step 1: Transcribe the audio
5 base_url = "https://api.assemblyai.com"
6 
7 headers = {
8   "authorization": "<YOUR_API_KEY>"
9 }
10 
11 # You can use a local filepath:
12 # with open("./my-audio.mp3", "rb") as f:
13 # response = requests.post(base_url + "/v2/upload",
14 # headers=headers,
15 # data=f)
16 # upload_url = response.json()["upload_url"]
17 # Or use a publicly-accessible URL:
18 
19 upload_url = "https://assembly.ai/sports_injuries.mp3"
20 
21 data = {
22   "audio_url": upload_url
23 }
24 
25 response = requests.post(base_url + "/v2/transcript", headers=headers, json=data)
26 
27 transcript_id = response.json()["id"]
28 polling_endpoint = base_url + f"/v2/transcript/{transcript_id}"
29 
30 while True:
31   transcript = requests.get(polling_endpoint, headers=headers).json()
32 
33   if transcript["status"] == "completed":
34     break
35 
36   elif transcript["status"] == "error":
37     raise RuntimeError(f"Transcription failed: {transcript['error']}")
38 
39   else:
40     time.sleep(3)
41 
42 # Step 2: Send transcript to LLM Gateway
43 prompt = "Provide a brief summary of the transcript."
44 
45 llm_gateway_data = {
46   "model": "claude-sonnet-4-5-20250929",
47   "messages": [
48     {"role": "user", "content": f"{prompt}\n\nTranscript: {transcript['text']}"}
49   ],
50   "max_tokens": 1000
51 }
52 
53 response = requests.post(
54   "https://llm-gateway.assemblyai.com/v1/chat/completions",
55   headers=headers,
56   json=llm_gateway_data
57 )
58 print(response.json()["choices"][0]["message"]["content"])

If you run the code above, you’ll see the following output:

1 The transcript describes several common sports injuries - runner's knee,
2 sprained ankle, meniscus tear, rotator cuff tear, and ACL tear. It provides
3 definitions, causes, and symptoms for each injury. The transcript seems to be
4 narrating sports footage and describing injuries as they occur to the athletes.
5 Overall, it provides an overview of these common sports injuries that can result
6 from overuse or sudden trauma during athletic activities

Before you begin

To complete this tutorial, you need:

Python, Node, .NET, Ruby or PHP installed.
An AssemblyAI account with a credit card set up.
Basic understanding of how to Transcribe an audio file.

Step 1: Install prerequisites

Python

JavaScript

C#

Ruby

PHP

Install the package via pip:

$ pip install requests

Step 2: Transcribe an audio file

LLM Gateway uses transcript text as input to generate text output. In this step, you’ll transcribe an audio file that you can later use with LLM Gateway.

For more information about transcribing audio, see Transcribe an audio file.

Python

JavaScript

C#

Ruby

PHP

1 import requests
2 import time
3 
4 base_url = "https://api.assemblyai.com"
5 
6 headers = {"authorization": "<YOUR_API_KEY>"}
7 
8 # You can use a local filepath:
9 # with open("./my-audio.mp3", "rb") as f:
10 #     response = requests.post(base_url + "/v2/upload", headers=headers, data=f)
11 #     upload_url = response.json()["upload_url"]
12 
13 # Or use a publicly-accessible URL:
14 upload_url = "https://assembly.ai/sports_injuries.mp3"
15 
16 data = {"audio_url": upload_url}
17 
18 response = requests.post(base_url + "/v2/transcript", headers=headers, json=data)
19 
20 transcript_id = response.json()["id"]
21 polling_endpoint = base_url + f"/v2/transcript/{transcript_id}"
22 
23 while True:
24     transcript = requests.get(polling_endpoint, headers=headers).json()
25 
26     if transcript["status"] == "completed":
27         break
28 
29     elif transcript["status"] == "error":
30         raise RuntimeError(f"Transcription failed: {transcript['error']}")
31 
32     else:
33         time.sleep(3)

Use existing transcript

If you’ve already transcribed an audio file you want to use, you can get an existing transcript using its ID. You can find the ID for previously transcribed audio files in the Processing queue.

Python

JavaScript

C#

Ruby

PHP

1 transcript = requests.get("https://api.assemblyai.com/v2/transcript/YOUR_TRANSCRIPT_ID", headers=headers).json()

Step 3: Send transcript to LLM Gateway

In this step, you’ll send the transcript text to LLM Gateway along with a prompt to generate text output.

The prompt is a text string that provides the LLM with instructions on how to generate the text output. You’ll combine the prompt with the transcript text and send it to LLM Gateway using the chat completions API.

Python

JavaScript

C#

Ruby

PHP

Write a prompt with instructions on how the LLM should generate the text output.

1 prompt = "Provide a brief summary of the transcript."

Python

JavaScript

C#

Ruby

PHP

Send the transcript text and prompt to LLM Gateway. The model parameter defines which LLM to use. For available models, see LLM Gateway Overview.

1 llm_gateway_data = {
2   "model": "claude-sonnet-4-5-20250929",
3   "messages": [
4     {"role": "user", "content": f"{prompt}\n\nTranscript: {transcript['text']}"}
5   ],
6   "max_tokens": 1000
7 }
8 
9 result = requests.post(
10   "https://llm-gateway.assemblyai.com/v1/chat/completions",
11   headers=headers,
12   json=llm_gateway_data
13 )

Print the result.

Python

JavaScript

C#

Ruby

PHP

1 print(result.json()["choices"][0]["message"]["content"])

The output will look something like this:

 The transcript describes several common sports injuries - runner's knee,
 sprained ankle, meniscus tear, rotator cuff tear, and ACL tear. It provides
 definitions, causes, and symptoms for each injury. The transcript seems to be
 narrating sports footage and describing injuries as they occur to the athletes.
 Overall, it provides an overview of these common sports injuries that can
 result from overuse or sudden trauma during athletic activities

Next steps

In this tutorial, you’ve learned how to generate LLM output based on your audio transcripts using LLM Gateway. The type of output depends on your prompt, so try exploring different prompts to see how they affect the output. Here’s a few more prompts to try.

“Provide an analysis of the transcript and offer areas to improve with exact quotes.”
“What’s the main take-away from the transcript?”
“Generate a set of action items from this transcript.”

To learn more about LLM Gateway and working with different models, see the following resources:

Need some help?

If you get stuck, or have any other questions, we’d love to help you out. Contact our support team at support@assemblyai.com or create a support ticket.