Monitor your SpeechAI app with OpenLIT

As AI technology continues to grow, the need for accurate, reliable, and scalable generative AI applications becomes increasingly important. These applications additionally require thorough monitoring to ensure they run smoothly and meet performance expectations.

In this blog post you'll learn why monitoring is essential for generative AI applications, and how you can add it to your AssemblyAI app via OpenLIT, a OpenTelemetry-native tool that provides developers with one-click observability.

Why You Should Monitor Your AI Applications

AI systems are increasingly becoming critical infrastructure for modern businesses, making robust monitoring not just a technical necessity but a strategic imperative. Here are just a few reasons your AI apps should incorporate monitoring:

Performance Optimization

Comprehensive monitoring provides deep visibility into your AI application's performance metrics. By tracking response times, resource utilization, and system behavior under varying loads, you can proactively identify and address bottlenecks, ensuring optimal user experience and system efficiency.

Diagnostic Intelligence

Without sophisticated tracing mechanisms, troubleshooting AI systems can become an exercise in speculation. Proper tracing illuminates can provide insight into what are often, or often seem to be, black-box solutions. Monitoring creates a coherent narrative of system behavior, from initial API calls to final output generation.

Accuracy and Reliability

AI systems require constant vigilance to maintain their performance standards, especially given that AI systems often suffer from model drift. Through systematic monitoring, you can track output consistency, detect subtle degradation patterns, and implement corrective measures before users experience any impact.

User Experience Enhancement

Monitoring user interactions provides invaluable insights into how your AI application serves its intended purpose. By analyzing usage patterns, feedback metrics, and engagement data, you can drive informed improvements to features and capabilities, resulting in enhanced user satisfaction and adoption.

Scalability Management

As demand for AI services grows, your infrastructure must evolve accordingly. Effective monitoring helps identify scaling triggers, capacity constraints, and resource utilization patterns, enabling informed decisions about when and how to scale your application while maintaining performance standards.

What is OpenTelemetry?

OpenTelemetry is an open standard for monitoring and observability, maintained by a global community of developers and experts. It provides a unified framework for collecting, processing, and exporting telemetry data across diverse software systems. Building on OpenTelemetry ensures your monitoring infrastructure leverages industry-proven standards that evolve with technological advances.

As a vendor-neutral solution, OpenTelemetry offers the freedom to switch between different monitoring backends without vendor lock-in. This flexibility allows you to adapt your observability stack as your needs change, while maintaining consistent instrumentation across your entire application landscape.

What is OpenLIT?

OpenLIT is an OpenTelemetry-native tool that simplifies the observability setup process for AI applications. Because OpenLIT is built on OpenTelemetry, you can easily transmit traces and metrics to popular tools like Grafana and New Relic. This seamless integration allows you to incorporate OpenLIT into your existing monitoring setup with one line of code.

How to monitor your AssemblyAI application with OpenLIT

We'll build a minimal example of a script that calls the AssemblyAI API for speech-to-text and prints out the resulting transcript. You'll need to have Python installed on your system to follow along.

First, install the AssemblyAI Python SDK, as well as the OpenLIT SDK:

pip install -U assemblyai openlit

If you don't already have an AssemblyAI key, you can get one for free here. Create a script called main.py, and then add the following code:

import assemblyai as aai

aai.settings.api_key = "YOUR_API_KEY"

transcriber = aai.Transcriber()
transcript = transcriber.transcribe("https://assembly.ai/wildfires.mp3")

print(transcript.text)

Make sure to replace YOUR_API_KEY with your AssemblyAI key that you can find on the homepage of your dashboard. While we set the API key inline here for simplicity, you should store it securely in a configuration file or environment variable in production code, and make sure to never check it into source control.

This script transcribes an example audio file and then prints the text to the console. Now let's add OpenLIT telemetry. Import the openlit library and add the  line openlit.init() to initialize OpenLit.

import assemblyai as aai
import openlit

aai.settings.api_key = "YOUR_API_KEY"
openlit.init()

transcriber = aai.Transcriber()
transcript = transcriber.transcribe("https://assembly.ai/wildfires.mp3")

print(transcript.text)

The OpenLIT SDK automatically logs all OpenTelemetry traces and metrics to the terminal, which can be useful for early development and debugging.

Once you're ready to store these traces and metrics for further analysis, configure the OpenTelemetry endpoint by configuring the following environment variables:

export OTEL_EXPORTER_OTLP_ENDPOINT="YOUR_OPENTELEMETRY_HTTP_ENDPOINT"
export OTEL_EXPORTER_OTLP_HEADERS="YOUR_OPENTELEMETRY_AUTH_HEADER"

Now that your app is continuously collecting metrics, you can go to your observability platform to find patterns, identify bottlenecks, and optimize your application based on the insights you've gained.

Next Steps

Monitoring is an integral part of developing and maintaining AI applications. It ensures that your applications are running smoothly, reliably, and efficiently. Integrating AssemblyAI with OpenLIT gives you a powerful combination of tools that provide robust, one-click observability. If you want to learn how you can self-host the OpenLIT UI yourself, check out the OpenLIT Docs.

To learn more about how to use our API and the features it offers, check out our Docs, or check out our cookbooks repository to browse solutions for common use cases. Alternatively, check out our blog for tutorials and deep-dives on AI theory, like this Introduction to LLMs, or our YouTube channel for project tutorials and more, like this one on building an AI voice agent in Python with DeepSeek R1:

‍