Models & features

Key Phrases

Global Englishen
Australian Englishen_au
British Englishen_uk
US Englishen_us

Universal-3-Prouniversal-3-pro
Universal-2universal-2

US & EU

The Key Phrases model identifies significant words and phrases in your transcript and lets you extract the most important concepts or highlights from your audio or video file.

Key phrase results are not embedded inline in the transcript text field. They are returned in a separate auto_highlights_result object in the API response, which contains each phrase along with its relevancy rank, occurrence count, and timestamps. See the Response section and code examples below for how to access this field.

Quickstart

Enable Key Phrases by setting auto_highlights to True in the JSON payload.

1import requests
2import time
3
4base_url = "https://api.assemblyai.com"
5
6headers = {
7    "authorization": "<YOUR_API_KEY>"
8}
9
10with open("./local_file.mp3", "rb") as f:
11    response = requests.post(base_url + "/v2/upload",
12                            headers=headers,
13                            data=f)
14
15upload_url = response.json()["upload_url"]
16
17data = {
18    "audio_url": upload_url, # You can also use a URL to an audio or video file on the web
19    "speech_models": ["universal-3-pro", "universal-2"],
20    "language_detection": True,
21    "auto_highlights": True
22}
23
24url = base_url + "/v2/transcript"
25response = requests.post(url, json=data, headers=headers)
26
27transcript_id = response.json()['id']
28polling_endpoint = base_url + "/v2/transcript/" + transcript_id
29
30print(f"Transcript ID:", transcript_id)
31
32while True:
33    transcription_result = requests.get(polling_endpoint, headers=headers).json()
34
35    if transcription_result['status'] == 'completed':
36        for result in transcription_result['auto_highlights_result']['results']:
37            print(f"Highlight: {result['text']}, Count: {result['count']}, Rank: {result['rank']}, Timestamps: {result['timestamps']}")
38        break
39    elif transcription_result['status'] == 'error':
40        raise RuntimeError(f"Transcription failed: {transcription_result['error']}")
41    else:
42        time.sleep(3)

Example output

1Highlight: air quality alerts, Count: 1, Rank: 0.08, Timestamps: [Timestamp(start=3978, end=5114)]
2Highlight: wide ranging air quality consequences, Count: 1, Rank: 0.08, Timestamps: [Timestamp(start=235388, end=238838)]
3Highlight: more fires, Count: 1, Rank: 0.07, Timestamps: [Timestamp(start=184716, end=185186)]
4...

API reference

Request

$curl https://api.assemblyai.com/v2/transcript \
>--header "Authorization: <YOUR_API_KEY>" \
>--header "Content-Type: application/json" \
>--data '{
>  "audio_url": "YOUR_AUDIO_URL",
>  "auto_highlights": true
>}'
KeyTypeDescription
auto_highlightsbooleanEnable Key Phrases.

Response

KeyTypeDescription
auto_highlights_resultobjectThe result of the Key Phrases model.
auto_highlights_result.statusstringIs either success or unavailable in the rare case that the Key Phrases model failed.
auto_highlights_result.resultsarrayA temporally-sequential array of key phrases.
auto_highlights_result.results[i].countnumberThe total number of times the i-th key phrase appears in the audio file.
auto_highlights_result.results[i].ranknumberThe total relevancy to the overall audio file of this key phrase. A greater number means that the key phrase is more relevant.
auto_highlights_result.results[i].textstringThe text itself of the key phrase.
auto_highlights_result.results[i].timestamps[j].startnumberThe starting time of the j-th appearance of the i-th key phrase.
auto_highlights_result.results[i].timestamps[j].endnumberThe ending time of the j-th appearance of the i-th key phrase.

The response also includes the request parameters used to generate the transcript.

Frequently Asked Questions

The Key Phrases model uses natural language processing and machine learning algorithms to analyze the frequency and distribution of words and phrases in your transcription. The algorithm identifies key phrases based on their relevancy score, which takes into account factors such as the number of times a phrase occurs, the distance between occurrences, and the overall length of the transcription.

The Key Phrases model is designed to identify important phrases and words in your transcription, whereas the Topic Detection model is designed to categorize your transcription into predefined topics. While both models use natural language processing and machine learning algorithms, they have different goals and approaches to analyzing your text.

Yes, the Key Phrases model can handle misspelled or unrecognized words to some extent. However, the accuracy of the detection may depend on the severity of the misspelling or the obscurity of the word. It’s recommended to provide high-quality, relevant audio files with accurate transcriptions for the best results.

Some limitations of the Key Phrases model include its limited understanding of context, which may lead to inaccuracies in identifying the most important phrases in certain cases, such as text with heavy use of jargon or idioms. Additionally, the model assigns higher scores to words or phrases that occur more frequently in the text, which may lead to an over-representation of common words and phrases that may not be as important in the context of the text. Finally, the Key Phrases model is a general-purpose algorithm that can’t be easily customized or fine-tuned for specific domains, meaning it may not perform as well for specialized texts where certain keywords or concepts may be more important than others.

To optimize the performance of the Key Phrases model, it’s recommended to provide high-quality, relevant audio files with accurate transcriptions, to review and adjust the model’s configuration parameters, such as the confidence threshold for key phrase detection, and to refer to the list of identified key phrases to guide the analysis. It may also be helpful to consider adding additional training data to the model or consulting with AssemblyAI support for further assistance.