In this tutorial, you'll build a server application in Go that transcribes an incoming Twilio phone call into text in real-time using direct API calls to AssemblyAI's Universal-Streaming API.
You'll use a Twilio MediaStream to stream the voice data to your local server application. You'll pass the voice data to AssemblyAI's Universal-Streaming Speech-to-Text API to transcribe the call into text, and then print it in your terminal in real-time.
This tutorial walks you through the code, step-by-step, but you can also find the complete source code at the end of this page or on the GitHub repo here.
Before you get started
To complete this tutorial, you'll need:
- An AssemblyAI account
- Signing up gets you $50 in free credits and access to Universal-Streaming by default!
- Go installed
- A Twilio account with a phone number
Step 1: Set up ngrok
Twilio will need to access your server through a publicly available URL. In this tutorial, you'll use ngrok to create a publicly available URL for an application running on your local computer.
If you already have a preferred way to expose your application publicly, you may skip this step.
- Sign up for an ngrok account
- Install ngrok for your platform
- Authenticate your ngrok agent using Your Authtoken:
Open an ngrok tunnel for port `8080`. ngrok will only tunnel connections while the following command is running:
You'll see something similar to the output below, where the URL next to Forwarding is the publicly available URL that forwards to your local 8080 port (https://84c5df474.ngrok-free.dev in the example output).
Copy the `Forwarding` URL in your terminal output and save it for the next step.
Step 2: Set up Twilio and open a ngrok tunnel
You'll need to register a phone number with Twilio and configure it to call your server application whenever someone calls that number.
- Sign up for a Twilio account
- To get a Twilio number, go to your Twilio console and go to `Phone Number > Manage > Buy a number`. There, you will see a list of numbers you can purchase for a small monthly fee - select one and click `Buy`. Note that we only need Voice capabilities for this tutorial.
Navigate to `Phone Numbers > Manage > Active numbers` and select the phone number you bought above. In the `Voice Configuration`, set a `Webhook` for when a call comes in, pasting the ngrok URL you just copied (from the previous step) under URL and setting the HTTP method to HTTP `POST`:
Then scroll down and click `Save Configuration` to save this change.
You have now configured your Twilio number to send a POST request to the ngrok URL when your number is called, and opened a tunnel that forwards this ngrok URL to port `8080` on your local machine.
We now have a working Go application that can successfully receive and respond to a Twilio phone call - it’s time to add in the WebSocket that receives incoming speech.
Step 3: Create a WebSocket server for Twilio media streams
In this step, you'll set up the Go server application to accept the Twilio MediaStream and connect to AssemblyAI's Universal-Streaming Speech-to-Text API.
Create and navigate into a new project directory:
Initialize your Go module:
Install the required dependencies:
Create a new file called main.go with the following content:
Create a `.env` file in your project directory and add your AssemblyAI API key:
Step 4: Define message structures
AssemblyAI's Universal-Streaming API uses specific message formats. Add these struct definitions to handle the different message types:
Step 5: Implement TwiML response handler
When someone calls the phone number, Twilio makes an HTTP request to an endpoint where you define how to respond. You'll use TwiML to define the instructions that tell Twilio what to do when you receive an incoming call.
Update your twilio function:
The TwiML instructs Twilio to:
- Say a greeting message to the caller
- Connect to a WebSocket stream at the /media endpoint
Step 6: Create audio buffering system
Since Twilio sends very small audio chunks (around 20ms each) but AssemblyAI's Universal-Streaming API requires chunks between 50-1000ms, we need to buffer the audio. Add this audio buffer implementation:
Step 7: Create transcript display system
To handle clean display of partial and final transcripts, add this display manager:
Step 8: Connect to AssemblyAI Universal-Streaming
Now implement the WebSocket connection to AssemblyAI's Universal-Streaming Speech-to-Text API:
This function:
- Builds the WebSocket URL with the required parameters for Universal-Streaming (v3)
- Sets the sample rate to 8000 Hz (Twilio's format)
- Specifies PCM μ-law encoding (Twilio's audio format)
- Enables formatted turns for better transcript output
- Configures turn detection parameters for natural conversation flow
Step 9: Handle audio forwarding
Add the function to forward audio from Twilio to AssemblyAI:
Step 10: Handle AssemblyAI responses
Implement the function to process responses from AssemblyAI:
This function handles three types of messages from AssemblyAI:
- Begin: Session establishment confirmation
- Turn: Transcript data (partial and final)
- Termination: Session end notification
The transcript handling logic ensures a smooth user experience by displaying partial transcripts in real-time and replacing them with final formatted versions.
Step 11: Implement the media WebSocket handler
Now implement the complete media function that ties everything together:
Step 12: Add session management
For robustness, add a session manager to track the AssemblyAI connection state (optional):
Step 13: Test your application
Now you're ready to test the complete application:
Start the server:
You should see output like:
Call your Twilio phone number. If prompted by your operating system, allow the application to access the network.
Once instructed by the voice, start speaking to see your call transcribed in your server log:
Most of this logging can be useful for debugging purposes, but can be safely removed to show only the real-time transcript output.
Learn more
In this tutorial, you built a Go application that transcribes incoming phone calls in real-time using Twilio and AssemblyAI's Universal-Streaming Speech-to-Text API.
This implementation demonstrates how to:
- Handle WebSocket connections for real-time audio streaming
- Integrate with Twilio's MediaStream for phone call capture
- Process audio data in mu-law format from Twilio
- Connect directly to AssemblyAI's WebSocket API for streaming transcription
To keep up with more content like this, make sure you subscribe to our newsletter.
Complete source code
Also available via the GitHub repo here.
