Segment A Phone Call using LeMUR | AssemblyAI

In this guide we will show you how to use AssemblyAI’s LLM, LeMUR, to segment a phone call.

Quickstart

1 import assemblyai as aai
2 
3 aai.settings.api_key = "YOUR_API_KEY"
4 transcriber = aai.Transcriber()
5 audio_url = "YOUR_AUDIO_URL" # You can also provide a path to a local audio file
6 
7 transcript = transcriber.transcribe(audio_url)
8 
9 phases = [
10     "Introduction",
11     "Complaint",
12     "Resolution",
13     "Goodbye"
14 ]
15 
16 prompt = f'''
17 Analyze the following transcript of a phone call conversation and divide it into the following phases:
18 {', '.join(phases)}
19 
20 You will be given the transcript in the format of VTT captions.
21 
22 For each phase:
23 1. Identify the start and end timestamps (in seconds)
24 2. Provide a brief summary of what happened in that phase
25 
26 Format your response as a JSON object with the following structure:
27 {{
28     "phases": [
29         {{
30             "name": "Phase Name",
31             "start_time": start_time_in_seconds,
32             "end_time": end_time_in_seconds,
33             "summary": "Brief summary of the phase"
34         }},
35         ...
36     ]
37 }}
38 
39 Ensure that all parts of the conversation are covered by a phase, using "Other" for any parts that don't fit into the specified phases.
40 '''
41 
42 result = aai.Lemur().task(
43     input_text=transcript.export_subtitles_vtt(),
44     prompt=prompt,
45     final_model=aai.LemurModel.claude3_5_sonnet
46 )
47 
48 print(result.response)

Get Started

Before we begin, make sure you have an AssemblyAI account and an API key. You can sign up for an AssemblyAI account and get your API key from your dashboard. You will need to upgrade your account by adding a credit card to have access to LeMUR.

Find more details on the current LeMUR pricing in the AssemblyAI pricing page.

Step-by-Step Instructions

Install the SDK:

$ pip install assemblyai

Import the SDK and set your AssemblyAI API key.

1 import assemblyai as aai
2 
3 aai.settings.api_key = "API_KEY"

Use AssemblyAI to transcribe a file and save the transcript.

1 audio_url = "YOUR_AUDIO_URL"
2 
3 transcript = aai.Transcriber().transcribe(audio_url)

Define the phases you want to identify.

Here is an example of phases that can be used for customer support calls:

1 phases = [
2     "Introduction",
3     "Complaint",
4     "Resolution",
5     "Goodbye"
6 ]

Prompt LeMUR using the Task Endpoint, analyze the transcript to divide it into phases, and return the response. This is an example prompt, which you can modify to suit your specific requirements. See our documentation for more information about prompt engineering.

1 prompt = f'''
2 Analyze the following transcript of a phone call conversation and divide it into the following phases:
3 {', '.join(phases)}
4 
5 You will be given the transcript in the format of VTT captions.
6 
7 For each phase:
8 1. Identify the start and end timestamps (in seconds)
9 2. Provide a brief summary of what happened in that phase
10 
11 Format your response as a JSON object with the following structure:
12 {{
13     "phases": [
14         {{
15             "name": "Phase Name",
16             "start_time": start_time_in_seconds,
17             "end_time": end_time_in_seconds,
18             "summary": "Brief summary of the phase"
19         }},
20         ...
21     ]
22 }}
23 
24 Ensure that all parts of the conversation are covered by a phase, using "Other" for any parts that don't fit into the specified phases.
25 '''
26 
27 result = aai.Lemur().task(
28     input_text=transcript.export_subtitles_vtt(),
29     prompt=prompt,
30     final_model=aai.LemurModel.claude3_5_sonnet
31 )
32 
33 print(result.response)

Example output of the analysis of a transcript divided into phases and formatted as a JSON object:

{
    "phases": [
        {
            "name": "Introduction",
            "start_time": 1.52,
            "end_time": 15.57,
            "summary": "The customer service representative greets the caller and asks how they can help. The caller states they want to know the status of their order refund."
        },
        {
            "name": "Complaint",
            "start_time": 15.57,
            "end_time": 59.41,
            "summary": "The representative asks for the order ID, which the caller provides. The representative confirms the order details and that it was cancelled. The caller mentions they couldn't complete their test."
        },
        {
            "name": "Resolution",
            "start_time": 59.41,
            "end_time": 210.01,
            "summary": "The representative informs the caller that the refund was initiated on April 8th and will be credited by April 21st. They explain the refund timeline and bank processing days. The caller expresses some confusion about the timeline, and the representative clarifies the process."
        },
        {
            "name": "Goodbye",
            "start_time": 210.01,
            "end_time": 235.8,
            "summary": "The caller accepts the explanation. The representative asks if there's anything else they can help with, requests feedback, and concludes the call with a farewell."
        }
    ]
}