Code Switching

Transcribe audio containing multiple languages with code switching detection. This feature enables accurate transcription of conversations where speakers naturally switch between languages during conversations.

Built-in code switching

Universal-3 Pro (Recommended)

Universal-3 Pro has built-in code switching capabilities across English, Spanish, Portuguese, French, German, and Italian. Set language_detection to True to automatically detect the spoken language, and use a prompt to preserve the original language mix in your transcript.

Here is an example of how Universal-3 Pro handles spoken audio in English and French.

Example output:

You literally lost your French? No, no, no. Mon français est là. L'italien, j'ai oublié, mais mon français est toujours là. Il partira jamais. Okay, but would you need a French coach? Could you consider having me? Oh yeah, yeah, absolutely, absolutely. Mais pour l'instant, le français est là, heureusement.

1
import requests
2
import time
3
4
base_url = "https://api.assemblyai.com"
5
headers = {"authorization": "<YOUR_API_KEY>"}
6
7
data = {
8
    "audio_url": "https://assemblyaiassets.com/audios/code_switching_multilingual.mp3",
9
    "language_detection": True,
10
    "speech_models": ["universal-3-pro", "universal-2"],
11
    "prompt": "The spoken language may change throughout the audio, transcribe in the original language mix (code-switching), preserving the words in the language they are spoken."
12
}
13
14
response = requests.post(base_url + "/v2/transcript", headers=headers, json=data)
15
16
if response.status_code != 200:
17
    print(f"Error: {response.status_code}, Response: {response.text}")
18
    response.raise_for_status()
19
20
transcript_response = response.json()
21
transcript_id = transcript_response["id"]
22
polling_endpoint = f"{base_url}/v2/transcript/{transcript_id}"
23
24
while True:
25
    transcript = requests.get(polling_endpoint, headers=headers).json()
26
    if transcript["status"] == "completed":
27
        print(transcript["text"])
28
        break
29
    elif transcript["status"] == "error":
30
        raise RuntimeError(f"Transcription failed: {transcript['error']}")
31
    else:
32
        time.sleep(3)

Universal-2

Supports code switching between 99 languages, with best performing code switching languages being en, es, de.

To enable code switching for Universal-2 only, set speech_models to universal-2, set language_detection as True, and set code_switching as True as shown in the example below.

1
import assemblyai as aai
2
3
aai.settings.api_key = "<YOUR_API_KEY>"
4
5
audio_file = "./bilingual-audio.mp3"
6
# audio_file = "https://assembly.ai/wildfires.mp3"
7
8
config = aai.TranscriptionConfig(
9
    speech_models=["universal-2"],
10
    language_detection=True,
11
    language_detection_options=aai.LanguageDetectionOptions(
12
      code_switching=True
13
    )
14
)
15
16
transcript = aai.Transcriber(config=config).transcribe(audio_file)
17
18
if transcript.status == "error":
19
  raise RuntimeError(f"Transcription failed: {transcript.error}")
20
21
print(transcript.text)

Example API Response

When enabling code switching with automatic language detection, the two detected language codes with the highest confidence and their confidence will be included in the transcript JSON.

1
"language_detection_results": {
2
      "code_switching_languages": [
3
           {"language": "en", "confidence": 0.8},
4
           {"language": "es", "confidence": 0.7}
5
     ]
6
}

99 languages coverage

To get the best performing transcription while extending code switching across all 99 supported languages, specify both “Universal-3 Pro” and “Universal-2” using the speech_models parameter, allowing our system to automatically route your audio based on language support.

Model routing behavior: The system attempts to use the models in priority order falling back to the next model when needed. For example, with ["universal-3-pro", "universal-2"], the system will try to use universal-3-pro for languages it supports (English, Spanish, Portuguese, French, German, and Italian), and automatically fall back to Universal-2 for all other languages. This ensures you get the best performing transcription where available while maintaining the widest language coverage.

Other supported languages

While additional languages are supported for code switching, optimal results typically require the non-English language to be dominant in the audio. For English-dominant content with other languages, standard single-language transcription may be more appropriate.

We highly recommend testing sample code switching files with your specific audio to assess performance and evaluate outputs. We also recommend using an LLM to correct and fine-tune our model’s outputs.

Manually Setting Language Codes

To manually set the language codes, you can use the language_codes parameter. A max of two language codes can be set and one code must be "en". For example, if your file contains both English and Spanish, it would be "language_codes": ["en", "es"].

1
import assemblyai as aai
2
3
aai.settings.api_key = "<YOUR_API_KEY>"
4
5
audio_file = "./bilingual-audio.mp3"
6
# audio_file = "https://assembly.ai/wildfires.mp3"
7
8
config = aai.TranscriptionConfig(
9
  speech_models=["universal-3-pro", "universal-2"],
10
  language_codes=["en", "es"]  # English-Spanish code switching
11
)
12
13
transcript = aai.Transcriber(config=config).transcribe(audio_file)
14
15
if transcript.status == "error":
16
  raise RuntimeError(f"Transcription failed: {transcript.error}")
17
18
print(transcript.text)

Code Switching Confidence Threshold

The code_switching_confidence_threshold parameter controls how the model routes transcription when multiple languages are detected. When code switching is enabled, the model detects up to two languages per audio file and assigns each a confidence score.

The threshold determines which language is used for transcription using the following logic:

• If the non-English language’s confidence score meets or exceeds the threshold, the audio is routed to that non-English language model.
• If the non-English language’s confidence score falls below the threshold, the audio is routed to whichever language has the highest overall confidence, which may be English or non-English.
• If both detected languages are non-English, the audio is always routed to whichever has the higher confidence score, regardless of the threshold.

1
import assemblyai as aai
2
3
aai.settings.api_key = "<YOUR_API_KEY>"
4
5
audio_file = "./bilingual-audio.mp3"
6
# audio_file = "https://assembly.ai/wildfires.mp3"
7
8
config = aai.TranscriptionConfig(
9
    speech_models=["universal-3-pro", "universal-2"],
10
    language_detection=True,
11
    language_detection_options=aai.LanguageDetectionOptions(
12
      code_switching=True,
13
      code_switching_confidence_threshold=0.5 # Optional parameter - this is set to 0.3 by default
14
    )
15
)
16
17
transcript = aai.Transcriber(config=config).transcribe(audio_file)
18
19
if transcript.status == "error":
20
  raise RuntimeError(f"Transcription failed: {transcript.error}")
21
22
print(transcript.text)

Troubleshooting

If your audio contains primarily one language with only occasional words from another language, standard single-language transcription may be more appropriate than code-switching mode.

Related features

• Supported Languages - View all available language options
• Speaker Diarization - Identify different speakers in multi-speaker conversations
• Keyterms Prompting - Boost transcription accuracy