Speaker identification and diarization with AssemblyAI

Add identification to existing transcripts

Sometimes you already have transcripts with speaker labels but need to add names later. This works well when speaker names aren't known during transcription.

import assemblyai as aai
import requests

aai.settings.api_key = "YOUR_API_KEY"

# Create transcript with diarization only first
config = aai.TranscriptionConfig(speaker_labels=True)
transcriber = aai.Transcriber()
transcript = transcriber.transcribe(
   "https://assembly.ai/wildfires.mp3",
   config=config
)

# Add speaker identification later
headers = {"authorization": "YOUR_API_KEY"}
understanding_body = {
   "transcript_id": transcript.id,
   "speech_understanding": {
       "request": {
           "speaker_identification": {
               "speaker_type": "name",
               "known_values": ["Michel Martin", "Peter DeCarlo"]
           }
       }
   }
}

# Send to the speech understanding endpoint
response = requests.post(
   "https://llm-gateway.assemblyai.com/v1/understanding",
   headers=headers,
   json=understanding_body
)

result = response.json()

# Process your identified speakers
for utterance in result["utterances"]:
   print(f"{utterance['speaker']}: {utterance['text']}")



This two-step process gives you flexibility. You can add identification anytime after transcription as long as the original transcript has speaker labels enabled.

Essential API parameters

You need to understand these key parameters for successful implementation:

Here's how to use them together:

config = aai.TranscriptionConfig(
   speaker_labels=True,
   speakers_expected=2,  # Optional: helps if you know exactly 2 speakers
   speech_understanding={
       "request": {
           "speaker_identification": {
               "speaker_type": "name",
               "known_values": ["John Smith", "Jane Doe"]
           }
       }
   }
)



Key requirements for your parameters:

• Speaker labels must be True: Without this, identification won't work
• Keep names under 35 characters: Longer names get truncated
• Match speaker type to your values: Use "name" for personal names, "role" for job titles

Understanding the response format

The response structure changes when identification is applied. Here's what you'll see before and after.

Without identification:

{
 "utterances": [
   {
     "speaker": "A",
     "text": "Welcome to today's interview.",
     "start": 240,
     "end": 2560,
     "words": [
       {
         "text": "Welcome",
         "start": 240,
         "end": 640,
         "speaker": "A"
       }
     ]
   }
 ]
}



With identification applied:

{
 "utterances": [
   {
     "speaker": "Michel Martin",
     "text": "Welcome to today's interview.",
     "start": 240,
     "end": 2560,
     "words": [
       {
         "text": "Welcome",
         "start": 240,
         "end": 640,
         "speaker": "Michel Martin"
       }
     ]
   }
 ]
}



Notice how the speaker field at both utterance and word level now contains the identified name instead of a generic label. This applies throughout your entire transcript.

Advanced implementation options

Identify speakers by role instead of names

Role-based identification works well when you don't know speaker names but understand their functions. This approach is perfect for customer service calls, medical consultations, or any scenario with defined roles.

Here's how to implement role-based identification:

import assemblyai as aai

aai.settings.api_key = "YOUR_API_KEY"

# Set up role-based identification
config = aai.TranscriptionConfig(
   speaker_labels=True,
   speech_understanding={
       "request": {
           "speaker_identification": {
               "speaker_type": "role",  # Changed from "name"
               "known_values": ["Agent", "Customer"]
           }
       }
   }
)

# Run transcription with role identification
transcriber = aai.Transcriber()
transcript = transcriber.transcribe(
   "your_call_center_audio.mp3",
   config=config
)

# Results show roles instead of names
for utterance in transcript.utterances:
   print(f"{utterance.speaker}: {utterance.text}")
   # Output: "Agent: How may I help you today?"
   # Output: "Customer: I need to check my order status."



Common role combinations you can use:

• Call Centers: ["Agent", "Customer"]
• Healthcare: ["Doctor", "Patient"]
• Interviews: ["Interviewer", "Interviewee"]
• Support: ["Support", "User"]
• Podcasts: ["Host", "Guest"]

Industry applications

Different industries use speaker identification and diarization for specific purposes:

Call center quality monitoring represents one of the most common use cases, with recent call center AI implementations demonstrating how financial services providers extract actionable insights from call transcripts. By separating agents and customers, supervisors can automatically analyze talk time ratios, interruption patterns, and script adherence.

Meeting intelligence platforms use name-based identification to track individual contributions and assign action items. When integrated with calendar systems, these platforms can automatically populate speaker names from meeting invites.

Healthcare documentation relies on role-based identification to maintain clear records of doctor-patient interactions, with studies showing clinical accuracy improvements of up to 60% when speakers are correctly identified. The separation enables automated extraction of symptoms, diagnoses, and treatment plans while maintaining compliance requirements.

Final words

Speaker identification and diarization with AssemblyAI simplifies complex audio processing into a single API workflow. You can enable both features together or add identification to existing transcripts, giving you flexibility for any use case.

AssemblyAI's Voice AI models provide reliable speech-to-text with deep speech understanding capabilities. The platform scales to handle enterprise workloads while maintaining accuracy across different audio conditions and speaker types.