Pre-recorded audio — AssemblyAI

Our Speech-to-Text model enables you to transcribe pre-recorded audio into written text.

On top of the transcription, you can enable other features and models, such as Speaker Diarization, by adding additional parameters to the same transcription request.

Choose model class

Choose between Best and Nano based on the cost and performance tradeoffs best suited for your application.

Quickstart

The following example transcribes an audio file from a URL.

Python

TypeScript

Go

Java

C#

Ruby

1 import assemblyai as aai
2 
3 aai.settings.api_key = "<YOUR_API_KEY>"
4 
5 # You can use a local filepath:
6 # audio_file = "./example.mp3"
7 
8 # Or use a publicly-accessible URL:
9 audio_file = (
10     "https://assembly.ai/wildfires.mp3"
11 )
12 
13 transcriber = aai.Transcriber()
14 
15 transcript = transcriber.transcribe(audio_file)
16 
17 if transcript.status == aai.TranscriptStatus.error:
18     print(f"Transcription failed: {transcript.error}")
19     exit(1)
20 
21 print(transcript.text)

Example output

1 Smoke from hundreds of wildfires in Canada is triggering air quality alerts
2 throughout the US. Skylines from Maine to Maryland to Minnesota are gray and
3 smoggy. And...

Word-level timestamps

The response also includes an array with information about each word:

Python

TypeScript

Go

Java

C#

Ruby

1 print(transcript.words)

1 [Word(text='Smoke', start=250, end=650, confidence=0.73033), Word(text='from', start=730, end=1022, confidence=0.99996), ...]

Transcript status

After you’ve submitted a file for transcription, your transcript has one of the following statuses:

Status	Description
`processing`	The audio file is being processed.
`queued`	The audio file is waiting to be processed.
`completed`	The transcription has completed successfully.
`error`	An error occurred while processing the audio file.

Handling errors

If the transcription fails, the status of the transcript is error, and the transcript includes an error property explaining what went wrong.

Python

TypeScript

Go

Java

C#

Ruby

1 if transcript.status == aai.TranscriptStatus.error:
2     print(f"Transcription failed: {transcript.error}")
3     exit(1)

A transcription may fail for various reasons:

Unsupported file format
Missing audio in file
Unreachable audio URL

If a transcription fails due to a server error, we recommend that you resubmit the file for transcription to allow another server to process the audio.

Select the speech model with Best and Nano

We use a combination of models to produce your results. You can select the class of models to use in order to make cost-performance tradeoffs best suited for your application. You can visit our pricing page for more information on our model tiers.

Python

TypeScript

Go

Java

C#

Ruby

Name	SDK Parameter	Description
Best (default)	`aai.SpeechModel.best`	Use our most accurate and capable models with the best results, recommended for most use cases.
Nano	`aai.SpeechModel.nano`	Use our less accurate, but much lower cost models to produce your results.

Python

TypeScript

Go

Java

C#

Ruby

You can change the model by setting the speech_model in the transcription config:

1 config = aai.TranscriptionConfig(speech_model=aai.SpeechModel.nano)
2 
3 transcriber = aai.Transcriber(config=config)

For a list of the supported languages for each model, see Supported languages.

Select the region

The default region is US, with base URL api.assemblyai.com. For EU data residency requirements, you can use our base URL for EU at api.eu.assemblyai.com.

The base URL for EU is currently only available for Async transcription.

Region	Base URL
US (default)	`api.assemblyai.com`
EU	`api.eu.assemblyai.com`

Python

TypeScript

Go

Java

C#

Ruby

To use the EU endpoint, set the base_url in the client settings like this:

1 aai.settings.base_url = "https://api.eu.assemblyai.com"

Automatic punctuation and casing

By default, the API automatically punctuates the transcription text and formats proper nouns, as well as converts numbers to their numerical form.

Python

TypeScript

Go

Java

C#

Ruby

To disable punctuation and text formatting, set punctuate and format_text to False in the transcription config.

1 # create a new TranscriptionConfig
2 config = aai.TranscriptionConfig(punctuate=False, format_text=False)
3 
4 # set the configuration
5 transcriber = aai.Transcriber(config=config)

Automatic language detection

Identify the dominant language spoken in an audio file and use it during the transcription. Enable it to detect any of the supported languages.

To reliably identify the dominant language, the file must contain at least 50 seconds of spoken audio.

Python

TypeScript

Go

Java

C#

Ruby

To enable it, set language_detection to True in the transcription config.

1 config = aai.TranscriptionConfig(language_detection=True)
2 
3 transcriber = aai.Transcriber(config=config)

Select model class based on detected language

By performing automatic language detection on a small chunk of audio first, you can then select between the Best or Nano model depending on the detected language. To learn more, see Separating automatic language detection from transcription.

Confidence score

If language detection is enabled, the API returns a confidence score for the detected language. The score ranges from 0.0 (low confidence) to 1.0 (high confidence).

Python

TypeScript

Go

Java

Ruby

C#

1 print(transcript.json_response["language_confidence"])

Set a language confidence threshold

You can set the confidence threshold that must be reached if language detection is enabled. An error will be returned if the language confidence is below this threshold. Valid values are in the range [0,1] inclusive.

Python

TypeScript

Go

Java

Ruby

1 config = aai.TranscriptionConfig(
2     language_detection=True,
3     language_confidence_threshold=0.4
4 )

Fallback to a default language

For a workflow that resubmits a transcription request using a default language if the threshold is not reached, see this cookbook.

Set language manually

If you already know the dominant language, you can use the language_code key to specify the language of the speech in your audio file.

Python

TypeScript

Go

Java

C#

Ruby

1 config = aai.TranscriptionConfig(language_code="es")

To see all supported languages and their codes, see Supported languages.

Custom spelling

Custom Spelling lets you customize how words are spelled or formatted in the transcript.

Python

TypeScript

Go

Java

C#

Ruby

To use Custom Spelling, pass a dictionary to set_custom_spelling() on the transcription config. Each key-value pair specifies a mapping from a word or phrase to a new spelling or format. The key specifies the new spelling or format, and the corresponding value is the word or phrase you want to replace.

1 config = aai.TranscriptionConfig()
2 config.set_custom_spelling(
3   {
4     "Gettleman": ["gettleman"],
5     "SQL": ["Sequel"],
6   }
7 )
8 transcriber = aai.Transcriber(config=config)

The key is case-sensitive, but the value isn’t. Additionally, the value can contain multiple words.

Custom vocabulary

To improve the transcription accuracy, you can boost certain words or phrases that appear frequently in your audio file.

To boost words or phrases, include the word_boost parameter in the transcription config.

You can also control how much weight to apply to each keyword or phrase. Include boost_param in the transcription config with a value of low, default, or high.

Python

TypeScript

Go

Java

C#

Ruby

1 config = aai.TranscriptionConfig(
2   word_boost=["aws", "azure", "google cloud"],
3   boost_param="high"
4 )
5 
6 transcriber = aai.Transcriber(config=config)

Follow formatting guidelines for custom vocabulary to ensure the best results:

Remove all punctuation except apostrophes.
Make sure each word is in its spoken form. For example, iphone seven instead of iphone 7.
Remove spaces between letters in acronyms.

Additionally, the model still accepts words with unique characters such as é, but converts them to their ASCII equivalent.

You can boost a maximum of 1,000 unique keywords and phrases, where each of them can contain up to 6 words.

Multichannel transcription

If you have a multichannel audio file with multiple speakers, you can transcribe each of them separately.

Python

TypeScript

Go

Java

C#

Ruby

To enable it, set multichannel to True in your transcription config.

1 config = aai.TranscriptionConfig(multichannel=True)
2 
3 transcriber = aai.Transcriber(config=config)
4 transcript = transcriber.transcribe(audio_url)
5 
6 print(transcript.json_response["audio_channels"])
7 
8 print(transcript.utterances)

Multichannel audio increases the transcription time by approximately 25%.

The response includes an audio_channels property with the number of different channels, and an additional utterances property, containing a list of turn-by-turn utterances.

Each utterance contains channel information, starting at 1.

Additionally, each word in the words array contains the channel identifier.

Dual-channel transcription

Use Multichannel instead.

Export SRT or VTT caption files

You can export completed transcripts in SRT or VTT format, which can be used for subtitles and closed captions in videos.

You can also customize the maximum number of characters per caption by specifying the chars_per_caption parameter.

Python

TypeScript

Go

Java

C#

Ruby

1 srt = transcript.export_subtitles_srt()
2 srt = transcript.export_subtitles_srt(chars_per_caption=32)
3 
4 vtt = transcript.export_subtitles_vtt()
5 vtt = transcript.export_subtitles_vtt(chars_per_caption=32)

Export paragraphs and sentences

You can retrieve transcripts that are automatically segmented into paragraphs or sentences, for a more reader-friendly experience.

The text of the transcript is broken down by either paragraphs or sentences, along with additional metadata.

Python

TypeScript

Go

Java

C#

Ruby

1 sentences = transcript.get_sentences()
2 for sentence in sentences:
3   print(sentence.text)
4 
5 paragraphs = transcript.get_paragraphs()
6 for paragraph in paragraphs:
7   print(paragraph.text)

The response is an array of objects, each representing a sentence or a paragraph in the transcript. See the API reference for more info.

Filler words

The following filler words are removed by default:

“um"
"uh"
"hmm"
"mhm"
"uh-huh"
"ah"
"huh"
"hm"
"m”

If you want to keep filler words in the transcript, you can set the disfluencies to true in the transcription config.

Python

TypeScript

Go

Java

C#

Ruby

1 config = aai.TranscriptionConfig(disfluencies=True)
2 
3 transcriber = aai.Transcriber(config=config)

Profanity filtering

You can automatically filter out profanity from the transcripts by setting filter_profanity to true in your transcription config.

Any profanity in the returned text will be replaced with asterisks.

Python

TypeScript

Go

Java

C#

Ruby

1 config = aai.TranscriptionConfig(filter_profanity=True)
2 
3 transcriber = aai.Transcriber(config=config)

Profanity filter isn’t perfect. Certain words may still be missed or improperly filtered.

Set the start and end of the transcript

If you only want to transcribe a portion of your file, you can set the audio_start_from and the audio_end_at parameters in your transcription config.

Python

TypeScript

Go

Java

C#

Ruby

1 config = aai.TranscriptionConfig(
2   audio_start_from=5000,  # The start time of the transcription in milliseconds
3   audio_end_at=15000  # The end time of the transcription in milliseconds
4 )
5 
6 transcriber = aai.Transcriber(config=config)

Speech threshold

To only transcribe files that contain at least a specified percentage of spoken audio, you can set the speech_threshold parameter. You can pass any value between 0 and 1.

If the percentage of speech in the audio file is below the provided threshold, the value of text is None and the response contains an error message:

1 Audio speech threshold 0.9461 is below the requested speech threshold value 1.0

Python

TypeScript

Go

Java

C#

Ruby

1 config = aai.TranscriptionConfig(speech_threshold=0.1)
2 
3 transcriber = aai.Transcriber(config=config)

Word search

You can search through a completed transcript for a specific set of keywords, which is useful for quickly finding relevant information.

The parameter can be a list of words, numbers, or phrases up to five words.

Python

TypeScript

Go

Java

C#

Ruby

1 # Set the words you want to search for
2 words = ["foo", "bar", "foo bar", "42"]
3 
4 matches = transcript.word_search(words)
5 
6 for match in matches:
7     print(f"Found '{match.text}' {match.count} times in the transcript")

Delete transcripts

You can remove the data from the transcript and mark it as deleted.

Python

TypeScript

Go

Java

C#

Ruby

1 aai.Transcript.delete_by_id("1234")

Account-level TTL value

Starting on 11-26-2024, the platform will assign an account-level Time to Live (TTL) for customers who have executed a Business Associate Agreement (BAA) with AssemblyAI. For those customers, all transcripts generated via the async transcription endpoint will be deleted after the TTL period.

As of the feature launch date:

The TTL is set to 3 days (subject to change).
Customers can still manually delete transcripts before the TTL period by using the deletion endpoint. However, they cannot keep transcripts on the platform after the TTL period has expired.

BAAs are limited to customers who process PHI, subject to HIPAA. If you are processing PHI and require a BAA, please reach out to sales@assemblyai.com.

API reference

You can find the API reference here.

Troubleshooting

How can I make certain words more likely to be transcribed?

You can include words, phrases, or both in the word_boost parameter. Any term included has its likelihood of being transcribed boosted.

Can I customize how words are spelled by the model?

Yes. The Custom Spelling feature gives you the ability to specify how words are spelled or formatted in the transcript text. For example, Custom Spelling could be used to change the spelling of all instances of the word “Ariana” to “Arianna”. It could also be used to change the formatting of “CS 50” to “CS50”.

Why am I receiving a "400 Bad Request" error when making an API request?

A “400 Bad Request” error typically indicates that there’s a problem with the formatting or content of the API request. Double-check the syntax of your request and ensure that all required parameters are included as described in the API reference. If the issue persists, contact our support team for assistance.

1	import assemblyai as aai
2
3	aai.settings.api_key = "<YOUR_API_KEY>"
4
5	# You can use a local filepath:
6	# audio_file = "./example.mp3"
7
8	# Or use a publicly-accessible URL:
9	audio_file = (
10	"https://assembly.ai/wildfires.mp3"
11	)
12
13	transcriber = aai.Transcriber()
14
15	transcript = transcriber.transcribe(audio_file)
16
17	if transcript.status == aai.TranscriptStatus.error:
18	print(f"Transcription failed: {transcript.error}")
19	exit(1)
20
21	print(transcript.text)

1	Smoke from hundreds of wildfires in Canada is triggering air quality alerts
2	throughout the US. Skylines from Maine to Maryland to Minnesota are gray and
3	smoggy. And...

1	config = aai.TranscriptionConfig(speech_model=aai.SpeechModel.nano)
2
3	transcriber = aai.Transcriber(config=config)

1	# create a new TranscriptionConfig
2	config = aai.TranscriptionConfig(punctuate=False, format_text=False)
3
4	# set the configuration
5	transcriber = aai.Transcriber(config=config)

1	config = aai.TranscriptionConfig(language_detection=True)
2
3	transcriber = aai.Transcriber(config=config)

1	config = aai.TranscriptionConfig(
2	language_detection=True,
3	language_confidence_threshold=0.4
4	)

1	config = aai.TranscriptionConfig()
2	config.set_custom_spelling(
3	{
4	"Gettleman": ["gettleman"],
5	"SQL": ["Sequel"],
6	}
7	)
8	transcriber = aai.Transcriber(config=config)

1	config = aai.TranscriptionConfig(
2	word_boost=["aws", "azure", "google cloud"],
3	boost_param="high"
4	)
5
6	transcriber = aai.Transcriber(config=config)

1	config = aai.TranscriptionConfig(multichannel=True)
2
3	transcriber = aai.Transcriber(config=config)
4	transcript = transcriber.transcribe(audio_url)
5
6	print(transcript.json_response["audio_channels"])
7
8	print(transcript.utterances)

1	srt = transcript.export_subtitles_srt()
2	srt = transcript.export_subtitles_srt(chars_per_caption=32)
3
4	vtt = transcript.export_subtitles_vtt()
5	vtt = transcript.export_subtitles_vtt(chars_per_caption=32)

1	sentences = transcript.get_sentences()
2	for sentence in sentences:
3	print(sentence.text)
4
5	paragraphs = transcript.get_paragraphs()
6	for paragraph in paragraphs:
7	print(paragraph.text)

1	config = aai.TranscriptionConfig(disfluencies=True)
2
3	transcriber = aai.Transcriber(config=config)

1	config = aai.TranscriptionConfig(filter_profanity=True)
2
3	transcriber = aai.Transcriber(config=config)

1	config = aai.TranscriptionConfig(
2	audio_start_from=5000, # The start time of the transcription in milliseconds
3	audio_end_at=15000 # The end time of the transcription in milliseconds
4	)
5
6	transcriber = aai.Transcriber(config=config)

1	# Set the words you want to search for
2	words = ["foo", "bar", "foo bar", "42"]
3
4	matches = transcript.word_search(words)
5
6	for match in matches:
7	print(f"Found '{match.text}' {match.count} times in the transcript")