The use of AI in business applications is exploding, with AI expected to boost U.S. Gross Domestic Product (GDP) by 2030. Generative AI tools like ChatGPT, which had over a million users within its first week of availability, have been especially popular.
Many companies across industries are racing to integrate AI into their tools and platforms, but are left wondering: what do I need to know before building AI-powered tools for businesses?
In this article, we’ll explore some of the top considerations product teams should address, including:
- Does the integration help consolidate my tech stack or expand it?
- Do I want to manage the AI model internally or have it managed for me?
- What kind of internal and external support would I need?
- What are my security and data privacy needs?
- Does the cost fit my needs now and for future use?
- Will the AI model or LLM and/or partner be able to grow with us?
What to consider when choosing an AI model, LLM or AI partner
If you’re looking to build with an AI model or Large Language Model (LLM), knowing which model or AI partner to choose can be challenging.
In addition, orchestrating the AI integration internally can be a large barrier to entry. Companies must balance input from executive leadership and product management teams, as well as unlock engineering and research resources to bring AI-powered tools and products to market.
This is what a traditional internal AI management plan might look like:
But the plan doesn’t need to be this complex. In fact, bringing an AI-powered tool or product to market can be achieved without so many steps.
The following six considerations should help your team more efficiently streamline your AI pipeline to speed time to deployment and boost return on investment in the long term.
1. Does the integration help consolidate my tech stack or expand it?
Building with AI doesn’t mean you have to invest in an inflated tech stack. In fact, finding the right AI partner may help you consolidate it instead.
For example, say your product team would like to build a conversation intelligence tool that records customer conversations, outputs an accurate transcript, summarizes the conversation, and automatically creates a list of follow-up items based on the topics covered.
Traditionally, this complex process could mean integrating different AI models, possibly from different providers, and engineering them to work together, as demonstrated in the image below:
Alternatively, using a framework like LeMUR that lets users leverage different LLMs and LLM capabilities can significantly streamline this process for your engineering team, condensing the multi-step process into a single API call:
Product teams should prioritize AI partners that can offer this condensed tech stack to reduce engineering load, as well as ease the integration process.
2. Do I want to manage the AI model internally or have it managed for me?
Companies looking to build with AI should also consider if they wish to manage the AI tech stack internally or externally.
Will the AI model or LLM be open-source and fine-tuned in-house or accessed via a third-party API?
For example, while large, closed-source LLMs are more expensive on the surface, open-source LLMs can come with a host of hidden costs related to infrastructure, engineering resources, and continued maintenance. Open-source LLMs and AI models will also require in-house AI research teams with extensive specialized AI knowledge in order to ensure that they are continually state-of-the-art and meet the continued needs of your customers.
3. What kind of internal or external support would I need?
You’ll also want to consider what kind of support your team will need if you decide to align with an AI provider. While many AI models and LLMs can be accessed directly from the research lab that produced them, some providers are not equipped to handle the extensive needs associated with enterprise-level companies.
You’ll want to thoroughly vet the AI provider to see how they will handle:
- Uptime: If your product relies on the AI model or LLM to function properly, you’ll need to ensure that your AI provider has exceptional standards for uptime, as well as clear lines of communication and next steps to follow in the event of an outage.
- Responsiveness: Consider how you will communicate with your AI provider—will you have a dedicated line of communication with quick responses 24/7? If not, are you equipped to handle any issues internally while you wait for a response?
- End-to-end support: AI, like many new technologies, comes with its learning curve. Will your AI provider be a sounding board that offers end-to-end support, from integration to deployment and beyond?
- Health checks: Regular health checks are an integral component of product uptime. Ensure that health checks are scheduled regularly to ensure any issues are caught and solved in advance.
4. What are my security and data privacy needs?
Building tools that process and analyze customer data will inevitably come with security and data privacy concerns. If you are partnering with an AI provider, make sure they can meet your compliance needs, whether it be HIPAA, GDPR, or other external or internal standards. Even if you choose to integrate an AI model or LLM yourself, you’ll want to understand how the model or LLM stores sensitive customer data to ensure security and compliance. Does the company that built the model use the data for training purposes? Ensure that you have a clear understanding of this before any commitment.
5. Does the cost fit my needs now and for future use?
You’ll also need to ensure that you have a complete understanding of the pricing model for the AI model or LLM you choose to integrate.
Many AI models, for example, are charged based on the size of the file being processed. Speech AI models, like those offered at AssemblyAI, are often charged by the hour/s of audio/video streams being processed.
LLMs are typically charged on input and output tokens. This article breaks down how tokens are determined, so you can have a better idea of what billing costs might look like.
Ensuring a complete understanding of the pricing model and how it will change as your company’s needs also change will ensure continued profitability.
6. Will the AI model, LLM, and/or partner be able to grow with us?
Inflexible AI models or LLMs will make it difficult to grow or modify your tools as needs arise. You should be able to increase data usage, add users, and make changes to your tools without significant effort. Ensure that your speed and usability will remain intact even as you make modifications and grow your customer base.
Read next: 7 best practices for product teams to consider when building with AI
