Transcribe streaming audio from a microphone in C# — AssemblyAI

Overview

By the end of this tutorial, you’ll be able to transcribe audio from your microphone in C#.

Supported languages

Streaming Speech-to-Text is only available for English.

Before you begin

To complete this tutorial, you need:

.NET 8 (earlier versions will work too with minor adjustments)
An AssemblyAI account with credit card set up.

Here’s the full sample code for what you’ll build in this tutorial:

1 using System.Diagnostics;
2 using System.Runtime.InteropServices;
3 using AssemblyAI.Realtime;
4 
5 // Set up the cancellation token, so we can stop the program with Ctrl+C
6 var cts = new CancellationTokenSource();
7 var ct = cts.Token;
8 Console.CancelKeyPress += (sender, e) => cts.Cancel();
9 
10 // Set up the realtime transcriber
11 const int sampleRate = 16_000;
12 await using var transcriber = new RealtimeTranscriber(new RealtimeTranscriberOptions
13 {
14     ApiKey = "<YOUR_API_KEY>",
15     SampleRate = sampleRate
16 });
17 
18 transcriber.SessionBegins.Subscribe(message => Console.WriteLine(
19     $"Session begins: \n- Session ID: {message.SessionId}\n- Expires at: {message.ExpiresAt}"
20 ));
21 transcriber.PartialTranscriptReceived.Subscribe(transcript =>
22 {
23     // don't do anything if nothing was said
24     if (string.IsNullOrEmpty(transcript.Text)) return;
25     Console.WriteLine($"Partial: {transcript.Text}");
26 });
27 transcriber.FinalTranscriptReceived.Subscribe(transcript =>
28 {
29     Console.WriteLine($"Final: {transcript.Text}");
30 });
31 transcriber.ErrorReceived.Subscribe(error => Console.WriteLine($"Real-time error: {error.Error}"));
32 transcriber.Closed.Subscribe(closeEvent =>
33     Console.WriteLine("Real-time connection closed: {0} - {1}",
34         closeEvent.Code,
35         closeEvent.Reason
36     )
37 );
38 
39 Console.WriteLine("Connecting to real-time transcript service");
40 
41 await transcriber.ConnectAsync().ConfigureAwait(false);
42 
43 Console.WriteLine("Starting recording");
44 
45 var soxArguments = string.Join(' ', [
46     // --default-device doesn't work on Windows
47     RuntimeInformation.IsOSPlatform(OSPlatform.Windows) ? "-t waveaudio default" : "--default-device",
48     "--no-show-progress",
49     $"--rate {sampleRate}",
50     "--channels 1",
51     "--encoding signed-integer",
52     "--bits 16",
53     "--type wav",
54     "-" // pipe
55 ]);
56 using var soxProcess = new Process
57 {
58     StartInfo = new ProcessStartInfo
59     {
60         FileName = "sox",
61         Arguments = soxArguments,
62         RedirectStandardOutput = true,
63         UseShellExecute = false,
64         CreateNoWindow = true
65     }
66 };
67 
68 soxProcess.Start();
69 var soxOutputStream = soxProcess.StandardOutput.BaseStream;
70 var buffer = new byte[4096];
71 while (await soxOutputStream.ReadAsync(buffer, 0, buffer.Length, ct) > 0)
72 {
73     if (ct.IsCancellationRequested) break;
74     await transcriber.SendAudioAsync(buffer);
75 }
76 
77 soxProcess.Kill();
78 await transcriber.CloseAsync();

Step 1: Set up a cancellation token

Set up a cancellation token so you can gracefully stop the application.

1 using System.Diagnostics;
2 using AssemblyAI.Realtime;
3 
4 // Set up the cancellation token, so we can stop the program with Ctrl+C
5 var cts = new CancellationTokenSource();
6 var ct = cts.Token;
7 Console.CancelKeyPress += (sender, e) => cts.Cancel();

The cancellation token will be cancelled when you press Ctrl+C.

Step 2: Install the AssemblyAI C# .NET SDK

Add the AssemblyAI NuGet package to your project:

.NET CLI

Package Manager

PackageReference

$ dotnet add package AssemblyAI

Step 3: Create a real-time transcriber

In this step, you’ll create a real-time transcriber and configure it to use your API key.

Browse to Account, and then click the text under Your API key to copy it.

Create a RealtimeTranscriber with your API key and a sample rate of 16 kHz. Replace YOUR_API_KEY with your copied API key.

1 // Set up the realtime transcriber
2 const int sampleRate = 16_000;
3 await using var transcriber = new RealtimeTranscriber(new RealtimeTranscriberOptions
4 {
5     ApiKey = "<YOUR_API_KEY>",
6     SampleRate = sampleRate
7 });

Sample rate

The sampleRate is the number of audio samples per second, measured in hertz (Hz). Higher sample rates result in higher quality audio, which may lead to better transcripts, but also more data being sent over the network.

We recommend the following sample rates:

Minimum quality: 8_000 (8 kHz)
Medium quality: 16_000 (16 kHz)
Maximum quality: 48_000 (48 kHz)

If you don’t set a sample rate on the real-time transcriber, it defaults to 16 kHz.

Subscribe to the different transcriber events and log the event parameters.

1 transcriber.SessionBegins.Subscribe(message => Console.WriteLine(
2     $"Session begins: \n- Session ID: {message.SessionId}\n- Expires at: {message.ExpiresAt}"
3 ));
4 transcriber.PartialTranscriptReceived.Subscribe(transcript =>
5 {
6     // don't do anything if nothing was said
7     if (string.IsNullOrEmpty(transcript.Text)) return;
8     Console.WriteLine($"Partial: {transcript.Text}");
9 });
10 transcriber.FinalTranscriptReceived.Subscribe(transcript =>
11 {
12     Console.WriteLine($"Final: {transcript.Text}");
13 });
14 transcriber.ErrorReceived.Subscribe(error => Console.WriteLine($"Real-time error: {error.Error}"));
15 transcriber.Closed.Subscribe(closeEvent =>
16     Console.WriteLine("Real-time connection closed: {0} - {1}",
17         closeEvent.Code,
18         closeEvent.Reason
19     )
20 );

The real-time transcriber returns two types of transcripts: partial and final.

Partial transcripts are returned as the audio is being streamed to AssemblyAI.
Final transcripts are returned when the service detects a pause in speech.

End of utterance controls

You can configure the silence threshold for automatic utterance detection and programmatically force the end of an utterance to immediately get a Final transcript.

Step 4: Connect the streaming service

Streaming Speech-to-Text uses WebSockets to stream audio to AssemblyAI. This requires first establishing a connection to the API.

1 Console.WriteLine("Connecting to real-time transcript service");
2 await transcriber.ConnectAsync();

Step 5: Record audio from microphone

In this step, you’ll use SoX, a cross-platform audio library, to record audio from your microphone.

Install SoX on your machine.

macOS

Windows

Linux

$ brew install sox

To run the SoX process and pipe the audio data to the process output, add the following code:

1 Console.WriteLine("Starting recording");
2 
3 var soxArguments = string.Join(' ', [
4     // --default-device doesn't work on Windows
5     RuntimeInformation.IsOSPlatform(OSPlatform.Windows) ? "-t waveaudio default" : "--default-device",
6     "--no-show-progress",
7     $"--rate {sampleRate}",
8     "--channels 1",
9     "--encoding signed-integer",
10     "--bits 16",
11     "--type wav",
12     "-" // pipe
13 ]);
14 using var soxProcess = new Process
15 {
16     StartInfo = new ProcessStartInfo
17     {
18         FileName = "sox",
19         Arguments = soxArguments,
20         RedirectStandardOutput = true,
21         UseShellExecute = false,
22         CreateNoWindow = true
23     }
24 };
25 
26 soxProcess.Start();

Audio data format

The SoX arguments configure the format of the audio output. The arguments configure the format to a single channel with 16-bit signed integer PCM encoding and 16 kHz sample rate.

If you want to stream data from elsewhere, make sure that your audio data is in the following format:

Single channel
16-bit signed integer PCM or mu-law encoding

By default, the Streaming STT service expects PCM16-encoded audio. If you want to use mu-law encoding, see Specifying the encoding.

Read the audio data from the SoX process output and send it to the real-time transcriber.

1 var soxOutputStream = soxProcess.StandardOutput.BaseStream;
2 var buffer = new byte[4096];
3 while (await soxOutputStream.ReadAsync(buffer, 0, buffer.Length, ct) > 0)
4 {
5     if (ct.IsCancellationRequested) break;
6     await transcriber.SendAudioAsync(buffer);
7 }

Step 6: Disconnect the real-time service

When you are done, stop the recording, and disconnect the transcriber to close the connection.

1 soxProcess.Kill();
2 await transcriber.CloseAsync();

Next steps

To learn more about Streaming Speech-to-Text, see the following resources:

Need some help?

If you get stuck, or have any other questions, we’d love to help you out. Contact our support team at support@assemblyai.com or create a support ticket.

1	using System.Diagnostics;
2	using System.Runtime.InteropServices;
3	using AssemblyAI.Realtime;
4
5	// Set up the cancellation token, so we can stop the program with Ctrl+C
6	var cts = new CancellationTokenSource();
7	var ct = cts.Token;
8	Console.CancelKeyPress += (sender, e) => cts.Cancel();
9
10	// Set up the realtime transcriber
11	const int sampleRate = 16_000;
12	await using var transcriber = new RealtimeTranscriber(new RealtimeTranscriberOptions
13	{
14	ApiKey = "<YOUR_API_KEY>",
15	SampleRate = sampleRate
16	});
17
18	transcriber.SessionBegins.Subscribe(message => Console.WriteLine(
19	$"Session begins: \n- Session ID: {message.SessionId}\n- Expires at: {message.ExpiresAt}"
20	));
21	transcriber.PartialTranscriptReceived.Subscribe(transcript =>
22	{
23	// don't do anything if nothing was said
24	if (string.IsNullOrEmpty(transcript.Text)) return;
25	Console.WriteLine($"Partial: {transcript.Text}");
26	});
27	transcriber.FinalTranscriptReceived.Subscribe(transcript =>
28	{
29	Console.WriteLine($"Final: {transcript.Text}");
30	});
31	transcriber.ErrorReceived.Subscribe(error => Console.WriteLine($"Real-time error: {error.Error}"));
32	transcriber.Closed.Subscribe(closeEvent =>
33	Console.WriteLine("Real-time connection closed: {0} - {1}",
34	closeEvent.Code,
35	closeEvent.Reason
36	)
37	);
38
39	Console.WriteLine("Connecting to real-time transcript service");
40
41	await transcriber.ConnectAsync().ConfigureAwait(false);
42
43	Console.WriteLine("Starting recording");
44
45	var soxArguments = string.Join(' ', [
46	// --default-device doesn't work on Windows
47	RuntimeInformation.IsOSPlatform(OSPlatform.Windows) ? "-t waveaudio default" : "--default-device",
48	"--no-show-progress",
49	$"--rate {sampleRate}",
50	"--channels 1",
51	"--encoding signed-integer",
52	"--bits 16",
53	"--type wav",
54	"-" // pipe
55	]);
56	using var soxProcess = new Process
57	{
58	StartInfo = new ProcessStartInfo
59	{
60	FileName = "sox",
61	Arguments = soxArguments,
62	RedirectStandardOutput = true,
63	UseShellExecute = false,
64	CreateNoWindow = true
65	}
66	};
67
68	soxProcess.Start();
69	var soxOutputStream = soxProcess.StandardOutput.BaseStream;
70	var buffer = new byte[4096];
71	while (await soxOutputStream.ReadAsync(buffer, 0, buffer.Length, ct) > 0)
72	{
73	if (ct.IsCancellationRequested) break;
74	await transcriber.SendAudioAsync(buffer);
75	}
76
77	soxProcess.Kill();
78	await transcriber.CloseAsync();

1	Console.WriteLine("Connecting to real-time transcript service");
2	await transcriber.ConnectAsync();