Extract Transcript Quotes with LeMURs Custom Text Input Parameter | AssemblyAI

This Colab will demonstrate how to use AssemblyAI’s LeMUR (Leveraging Large Language Models to Understand Recognized Speech) framework to process an audio file and find the best quotes included in it by sending in the timestamped transcript via LeMUR’s input_text parameter.

Quickstart

1 import assemblyai
2 
3 assemblyai.settings.api_key = "YOUR_API_KEY"
4 transcriber = assemblyai.Transcriber()
5 
6 transcript = transcriber.transcribe("https://github.com/AssemblyAI-Examples/audio-examples/raw/main/20230607_me_canadian_wildfires.mp3")
7 
8 result = assemblyai.Lemur().question(
9     input_text=f"{[(sentence.text, sentence.start, sentence.end) for sentence in transcript.get_sentences()]}",
10     final_model=assemblyai.LemurModel.claude3_5_sonnet,
11     questions=[
12         {
13             "question": "What are the most engaging quotes from this transcript?",
14             "context": "This is a list of sentences from the transcript, with each sentence having a start and end timestamp in milliseconds.",
15             "answer_format": "Exact quotes with the start and end timestamp of each sentence."
16         }
17     ]
18 )
19 
20 print(result.response[0].answer)

Getting Started

Before we begin, make sure you have an AssemblyAI account and an API key. You can sign up for an AssemblyAI account and get your API key from your dashboard.

Step-by-Step Instructions

First, let’s install the AssemblyAI SDK.

$ pip install -U assemblyai

Then we’ll import the SDK and set our AssemblyAI API key.

1 import assemblyai
2 
3 assemblyai.settings.api_key = "API_KEY_HERE"

Next, we’ll use AssemblyAI to transcribe a file and save our transcript for later use.

1 transcriber = assemblyai.Transcriber()
2 
3 transcript = transcriber.transcribe("https://github.com/AssemblyAI-Examples/audio-examples/raw/main/20230607_me_canadian_wildfires.mp3")

Then we’ll take the timestamped sentences array from our transcript and provide it to the LeMUR Q&A endpoint to extract the most engaging quotes from this transcript with their associated timestamps.

1 result = assemblyai.Lemur().question(
2     input_text=f"{[(sentence.text, sentence.start, sentence.end) for sentence in transcript.get_sentences()]}",
3     final_model=assemblyai.LemurModel.claude3_5_sonnet,
4     questions=[
5         {
6             "question": "What are the most engaging quotes from this transcript?",
7             "context": "This is a list of sentences from the transcript, with each sentence having a start and end timestamp in milliseconds.",
8             "answer_format": "Exact quotes with the start and end timestamp of each sentence."
9         }
10     ]
11 )
12 
13 print(result.response[0].answer)

Example response:

("So the concentrations of these particles in the air are just much, much higher than we typically see.", 113338, 119698)
("And exposure to those high levels can lead to a host of health problems.", 119784, 123314)
("I think New York has some of the higher concentrations right now, but that's going to change as that air moves away from the New York area.", 170950, 176930)
("Looking into the future, the fire season is starting earlier and lasting longer and we're seeing more frequent fires.", 245216, 251254)