LeMUR

Summarize your audio data

1package main
2
3import (
4 "context"
5 "fmt"
6
7 aai "github.com/AssemblyAI/assemblyai-go-sdk"
8)
9
10func main() {
11 ctx := context.Background()
12
13 client := aai.NewClient("<YOUR_API_KEY>")
14
15 // You can use a local file:
16 /*
17 f, err := os.Open("./example.mp3")
18 [error handling here]
19 transcript, err := client.Transcripts.TranscribeFromReader(ctx, f, params)
20 */
21
22 // Or use a publicly-accessible URL:
23 audioURL := "https://assembly.ai/sports_injuries.mp3"
24
25 transcript, _ := client.Transcripts.TranscribeFromURL(ctx, audioURL, nil)
26
27 prompt := "Provide a brief summary of the transcript."
28
29 var params aai.LeMURTaskParams
30 params.Prompt = aai.String(prompt)
31 params.TranscriptIDs = []string{aai.ToString(transcript.ID)}
32 params.FinalModel = "anthropic/claude-3-5-sonnet"
33
34 result, _ := client.LeMUR.Task(ctx, params)
35
36 fmt.Println(*result.Response)
37}

If you run the code above, you’ll see the following output:

1The transcript describes several common sports injuries - runner's knee,
2sprained ankle, meniscus tear, rotator cuff tear, and ACL tear. It provides
3definitions, causes, and symptoms for each injury. The transcript seems to be
4narrating sports footage and describing injuries as they occur to the athletes.
5Overall, it provides an overview of these common sports injuries that can result
6from overuse or sudden trauma during athletic activities

Ask questions about your audio data

Q&A with the task endpoint

To summarize the content in your audio data, define a summarization prompt and call client.LeMUR.Task(). Use the TranscriptIDs parameter to send one or more transcripts as additional context for the model.

1package main
2
3import (
4 "context"
5 "fmt"
6
7 aai "github.com/AssemblyAI/assemblyai-go-sdk"
8)
9
10func main() {
11 ctx := context.Background()
12
13 client := aai.NewClient("<YOUR_API_KEY>")
14
15 // Step 1: Transcribe an audio file. For local files see our Getting Started guides.
16 audioURL := "https://assembly.ai/sports_injuries.mp3"
17 transcript, _ := client.Transcripts.TranscribeFromURL(ctx, audioURL, nil)
18
19 // Step 2: Define a prompt with your question.
20 prompt := "What is a runner's knee?"
21
22 // Step 3: Apply LeMUR.
23 var params aai.LeMURTaskParams
24 params.Prompt = aai.String(prompt)
25 params.TranscriptIDs = []string{aai.ToString(transcript.ID)}
26 params.FinalModel = "anthropic/claude-3-5-sonnet"
27
28 result, _ := client.LeMUR.Task(ctx, params)
29
30 fmt.Println(*result.Response)
31}

#Example output

1Based on the transcript, runner's knee is a condition characterized
2by pain behind or around the kneecap. It is caused by overuse,
3muscle imbalance and inadequate stretching. Symptoms include pain
4under or around the kneecap and pain when walking.

Q&A with the question-answer endpoint

The LeMUR Question & Answer function requires no prompt engineering and facilitates more deterministic and structured outputs. See the code examples below for more information on how to use this endpoint.

To use it, define a list of questions. For each question, you can define additional context and specify either a answer_format or a list of answer_options. Additionally, you can define an overall context.

1package main
2
3import (
4 "context"
5 "fmt"
6
7 aai "github.com/AssemblyAI/assemblyai-go-sdk"
8)
9
10func main() {
11 ctx := context.Background()
12
13 client := aai.NewClient("<YOUR_API_KEY>")
14
15 audioURL := "https://assembly.ai/meeting.mp4"
16 transcript, err := client.Transcripts.TranscribeFromURL(ctx, audioURL, nil)
17
18 questions := []aai.LemurQuestion{
19 {
20 Question: "What are the top level KPIs for engineering?",
21 Context: aai.String("KPI stands for key performance indicator"),
22 AnswerFormat: aai.String("short sentence"),
23 },
24 {
25 Question: "How many days has it been since the data team has gotten updated metrics?",
26 AnswerOptions: &[]string{"1", "2", "3", "4", "5", "6", "7", "more than 7"},
27 },
28 }
29
30 var params aai.LemurQuestionParams
31 params.Questions = questions
32 params.TranscriptIDs = []string{aai.ToString(transcript.ID)}
33 params.Context = aai.String("A GitLab meeting to discuss logistics")
34 params.FinalModel = aai.String("anthropic/claude-3-5-sonnet")
35
36 result, err := client.LeMUR.Question(ctx, params)
37
38
39 for _, response := range result.Response {
40 fmt.Printf("Question: %s\n", response.Question)
41 fmt.Printf("Answer: %s\n\n", response.Answer)
42 }
43}

For the full API reference, as well as the supported models and FAQs, refer to the full LeMUR Q&A guide.

Change the model type

LeMUR features the following LLMs:

  • Claude 3.5 Sonnet
  • Claude 3 Opus
  • Claude 3 Haiku
  • Claude 3 Sonnet

You can switch the model by specifying the final_model parameter.

1var params aai.LeMURTaskParams
2params.Prompt = aai.String(prompt)
3params.TranscriptIDs = []string{aai.ToString(transcript.ID)}
4params.FinalModel = "anthropic/claude-3-5-sonnet"
5
6result, _ := client.LeMUR.Task(ctx, params)
ModelSDK ParameterDescription
Claude 3.5 Sonnet"anthropic/claude-3-5-sonnet"Claude 3.5 Sonnet is the most intelligent model to date, outperforming Claude 3 Opus on a wide range of evaluations, with the speed and cost of Claude 3 Sonnet. This uses Anthropic’s Claude 3.5 Sonnet model version claude-3-5-sonnet-20240620.
Claude 3.0 Opus"anthropic/claude-3-opus"Claude 3 Opus is good at handling complex analysis, longer tasks with many steps, and higher-order math and coding tasks.
Claude 3.0 Haiku"anthropic/claude-3-haiku"Claude 3 Haiku is the fastest model that can execute lightweight actions.
Claude 3.0 Sonnet"anthropic/claude-3-sonnet"Claude 3 Sonnet is a legacy model with a balanced combination of performance and speed for efficient, high-throughput tasks.

You can find more information on pricing for each model here.

Change the maximum output size

You can change the maximum output size in tokens by specifying the MaxOutputSize parameter. Up to 4000 tokens are allowed.

1var params aai.LeMURTaskParams
2params.Prompt = aai.String(prompt)
3params.TranscriptIDs = []string{aai.ToString(transcript.ID)}
4params.MaxOutputSize = aai.Int64(2000)
5
6result, _ := client.LeMUR.Task(ctx, params)

Change the temperature

You can change the temperature by specifying the temperature parameter, ranging from 0.0 to 1.0.

Higher values result in answers that are more creative, lower values are more conservative.

1var params aai.LeMURTaskParams
2params.Prompt = aai.String(prompt)
3params.TranscriptIDs = []string{aai.ToString(transcript.ID)}
4params.Temperature = aai.Float64(0.7)
5
6result, _ := client.LeMUR.Task(ctx, params)

Send customized input

You can submit custom text inputs to LeMUR without transcript IDs. This allows you to customize the input, for example, you could include the speaker labels for the LLM.

To submit custom text input, use the InputText parameter instead of TranscriptIDs.

1transcript, _ := client.Transcripts.TranscribeFromURL(ctx, audioURL, &aai.TranscriptOptionalParams{
2 SpeakerLabels: aai.Bool(true),
3})
4
5var textWithSpeakerLabels string
6
7for _, utterance := range transcript.Utterances {
8 textWithSpeakerLabels += fmt.Sprintf("Speaker %s:\n%s\n",
9 aai.ToString(utterance.Speaker),
10 aai.ToString(utterance.Text),
11 )
12}
13
14var params aai.LeMURTaskParams
15params.Prompt = aai.String(prompt)
16params.InputText = aai.String(textWithSpeakerLabels)
17
18result, _ := client.LeMUR.Task(ctx, params)

Submit multiple transcripts

LeMUR can easily ingest multiple transcripts in a single API call.

You can feed in up to a maximum of 100 files or 100 hours, whichever is lower.

1var params aai.LeMURTaskParams
2params.TranscriptIDs = []string{id1, id2, id3}
3params.Prompt = aai.String("Provide a summary of these customer calls.")
4
5result, _ := client.LeMUR.Task(ctx, params)

Delete LeMUR request data

You can delete the data for a previously submitted LeMUR request.

Response data from the LLM, as well as any context provided in the original request will be removed.

1result = transcript.lemur.task(prompt)
2
3deletion_response = aai.Lemur.purge_request_data(result.request_id)