---
title: "Voice AI"
description: "Compare voice AI platforms: speech-to-text and text-to-speech APIs, realtime speech models, voice agent frameworks like LiveKit, and phone agent platforms."
type: "platform-category"
canonical_url: "https://theaiengineering.com/platforms/voice-ai"
markdown_url: "https://theaiengineering.com/platforms/voice-ai.md"
topic: "AI agents"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
citation: "The AI Engineering Editors. \"Voice AI.\" The AI Engineering. https://theaiengineering.com/platforms/voice-ai"
---

# Voice AI

Voice AI platforms are the APIs, models, and frameworks used to build applications that listen and speak, from speech-to-text and text-to-speech services to complete voice agents that hold real-time conversations on the web or over the phone.

Speech model providers such as Deepgram, AssemblyAI, Speechmatics, Gladia, and Soniox focus on transcription, while ElevenLabs, Cartesia, Rime, Inworld, Fish Audio, and Murf focus on synthesis, and several now offer both. OpenAI, Google, and Amazon serve speech-to-speech models through the OpenAI Realtime API, the Gemini Live API, and Nova Sonic.

Open-source frameworks such as LiveKit Agents and Pipecat connect these services into low-latency pipelines that developers control in code. Hosted platforms such as Vapi, Retell AI, Bland, and Synthflow add telephony, call flows, and analytics for phone agents. Meta acquired PlayAI in 2025, Inworld acquired Ultravox in 2026, and LMNT has shut down.

## Voice AI compared

| Platform | Open source | Deployment | Best for |
| --- | --- | --- | --- |
| [Amazon Nova Sonic](https://aws.amazon.com/nova/models/) | No | API | Speech-to-speech agents on AWS |
| [AssemblyAI](https://www.assemblyai.com) | No | Managed cloud, API | Transcription plus speech understanding features |
| [Azure Speech in Foundry Tools](https://azure.microsoft.com/en-us/products/ai-foundry/tools/speech) | No | Managed cloud, API | Speech services inside the Azure ecosystem |
| [Bland](https://www.bland.ai) | No | Managed cloud, API, Self-hosted | Phone agents whose calls stay with one vendor |
| [Cartesia](https://www.cartesia.ai) | No | Managed cloud, API | Low-latency streaming speech for voice agents |
| [Deepgram](https://deepgram.com) | No | Managed cloud, API | Real-time transcription in production voice apps |
| [ElevenLabs](https://elevenlabs.io) | No | Managed cloud, API | Expressive voices and voice agents from one vendor |
| [Fish Audio](https://fish.audio) | No | Managed cloud, API, Self-hosted | Voice cloning with an option to self-host models |
| [Gemini Live API](https://ai.google.dev/gemini-api/docs/live) | No | API | Voice and video agents on Gemini models |
| [Gladia](https://www.gladia.io) | No | Managed cloud, API | Multilingual transcription with audio intelligence add-ons |
| [Hume AI](https://www.hume.ai) | No | Managed cloud, API | Analyzing and evaluating expressive speech |
| [Inworld](https://inworld.ai) | No | Managed cloud, API | Realtime voice for consumer applications |
| [LiveKit Agents](https://docs.livekit.io/agents/) | Yes (Apache-2.0) | Library, Managed cloud, Self-hosted | WebRTC voice agents with full code control |
| [Murf](https://murf.ai) | No | Managed cloud, API | Multilingual text-to-speech for agents and content |
| [OpenAI Realtime API](https://developers.openai.com/api/docs/guides/realtime) | No | API | Speech-to-speech agents on OpenAI models |
| [Pipecat](https://www.pipecat.ai) | Yes (BSD-2-Clause) | Library, Managed cloud, Self-hosted | Provider-agnostic voice pipelines in Python |
| [Retell AI](https://www.retellai.com) | No | Managed cloud, API, Self-hosted | Call automation built with a visual flow editor |
| [Rime](https://www.rime.ai) | No | Managed cloud, API | Conversational voices for phone and contact-center agents |
| [Soniox](https://soniox.com) | No | Managed cloud, API | Multilingual live transcription and translation |
| [Speechmatics](https://www.speechmatics.com) | No | Managed cloud, API, Self-hosted | Transcription with on-premises or on-device deployment |
| [Synthflow](https://synthflow.ai) | No | Managed cloud | Contact centers deploying voice agents |
| [TEN Framework](https://theten.ai) | No | Library, Self-hosted | Real-time voice agents alongside Agora services |
| [Twilio ConversationRelay](https://www.twilio.com/en-us/products/conversational-ai/conversationrelay) | No | Managed cloud, API | Adding an LLM to existing Twilio voice numbers |
| [Ultravox](https://www.ultravox.ai) | No | Managed cloud, API | Speech-native voice agents with telephony |
| [Vapi](https://vapi.ai) | No | Managed cloud, API | API-driven phone agents with swappable model providers |

## Profiles

### Amazon Nova Sonic

Amazon Nova 2.5 Sonic is Amazon's speech-to-speech model for building voice agents that hold natural, real-time conversations, with reasoning, instruction following, and tool calling so an agent can take actions during a call.

- Company: Amazon Web Services
- Website: https://aws.amazon.com/nova/models/
- Open source: No
- Deployment: API
- Best for: Speech-to-speech agents on AWS
- Permalink: https://theaiengineering.com/platforms/voice-ai#amazon-nova-sonic

### AssemblyAI

AssemblyAI provides speech AI models for speech-to-text on pre-recorded and streaming audio, with speaker detection, sentiment analysis, and other speech understanding features available through its API and SDKs.

- Website: https://www.assemblyai.com
- Open source: No
- Deployment: Managed cloud, API
- Languages: Python, TypeScript
- Best for: Transcription plus speech understanding features
- Permalink: https://theaiengineering.com/platforms/voice-ai#assemblyai

### Azure Speech in Foundry Tools

Azure Speech in Foundry Tools, formerly Azure AI Speech, provides speech recognition, text-to-speech, and customized speech models for multilingual applications, and it supplies transcription and synthesis for Voice Live agents in Microsoft Foundry.

- Company: Microsoft
- Website: https://azure.microsoft.com/en-us/products/ai-foundry/tools/speech
- Open source: No
- Deployment: Managed cloud, API
- Best for: Speech services inside the Azure ecosystem
- Permalink: https://theaiengineering.com/platforms/voice-ai#azure-speech

### Bland

Bland is an enterprise platform for AI phone agents that runs its own speech-to-text, language, and text-to-speech models on its own infrastructure, with on-premises options for sensitive workloads in industries such as healthcare and finance.

- Website: https://www.bland.ai
- Open source: No
- Deployment: Managed cloud, API, Self-hosted
- Best for: Phone agents whose calls stay with one vendor
- Permalink: https://theaiengineering.com/platforms/voice-ai#bland-ai

### Cartesia

Cartesia is an AI lab building real-time voice models and agents, offering streaming text-to-speech over WebSockets, real-time speech-to-text with its Ink models, and a platform for production voice agents.

- Website: https://www.cartesia.ai
- Open source: No
- Deployment: Managed cloud, API
- Languages: Python, TypeScript
- Best for: Low-latency streaming speech for voice agents
- Permalink: https://theaiengineering.com/platforms/voice-ai#cartesia

### Deepgram

Deepgram offers speech-to-text, text-to-speech, and Voice Agent APIs for enterprise voice applications, with real-time streaming, SDKs, and a command-line tool for transcribing audio and synthesizing speech.

- Website: https://deepgram.com
- Open source: No
- Deployment: Managed cloud, API
- Languages: Python, TypeScript
- Best for: Real-time transcription in production voice apps
- Permalink: https://theaiengineering.com/platforms/voice-ai#deepgram

### ElevenLabs

ElevenLabs provides APIs and SDKs for text-to-speech, speech-to-text, voice cloning, sound effects, and voice changing, plus a platform for conversational voice agents, with voices and models that cover many languages.

- Website: https://elevenlabs.io
- Headquarters: London, United Kingdom
- Founded: 2022
- Open source: No
- Deployment: Managed cloud, API
- Languages: Python, TypeScript
- Best for: Expressive voices and voice agents from one vendor
- Permalink: https://theaiengineering.com/platforms/voice-ai#elevenlabs

### Fish Audio

Fish Audio offers text-to-speech, voice cloning, and speech-to-text through an API with real-time streaming, and it publishes speech models under the Fish Audio Community License, which permits research and limited commercial use and self-hosting.

- Website: https://fish.audio
- GitHub: https://github.com/fishaudio/fish-speech
- Open source: No
- Deployment: Managed cloud, API, Self-hosted
- Best for: Voice cloning with an option to self-host models
- Permalink: https://theaiengineering.com/platforms/voice-ai#fish-audio

### Gemini Live API

The Gemini Live API enables low-latency, real-time voice and video interactions with Gemini models over a stateful WebSocket connection, with barge-in, audio transcriptions, and tool use including function calling and Google Search.

- Company: Google
- Website: https://ai.google.dev/gemini-api/docs/live
- Open source: No
- Deployment: API
- Best for: Voice and video agents on Gemini models
- Permalink: https://theaiengineering.com/platforms/voice-ai#gemini-live-api

### Gladia

Gladia is an audio transcription and intelligence API for voice products, providing real-time and pre-recorded speech-to-text plus translation, summarization, and entity extraction, with SDKs, a CLI, and an MCP server.

- Website: https://www.gladia.io
- Open source: No
- Deployment: Managed cloud, API
- Best for: Multilingual transcription with audio intelligence add-ons
- Permalink: https://theaiengineering.com/platforms/voice-ai#gladia

### Hume AI

Hume AI works on emotionally intelligent voice AI; its current developer platform centers on an Audio Decisions API that answers plain-language questions about audio and an Expression Measurement API, plus voice evaluation services.

- Website: https://www.hume.ai
- Open source: No
- Deployment: Managed cloud, API
- Best for: Analyzing and evaluating expressive speech
- Permalink: https://theaiengineering.com/platforms/voice-ai#hume-ai

### Inworld

Inworld provides voice APIs for text-to-speech, speech-to-text, and realtime conversations over WebSockets, plus inference APIs for text, image, and video models, and it acquired the Ultravox voice agent platform in 2026.

- Website: https://inworld.ai
- Open source: No
- Deployment: Managed cloud, API
- Best for: Realtime voice for consumer applications
- Permalink: https://theaiengineering.com/platforms/voice-ai#inworld-ai

### LiveKit Agents

LiveKit Agents is an open-source framework for realtime voice and multimodal agents that join LiveKit rooms as participants over WebRTC, with Python and Node.js SDKs, any STT, LLM, TTS, or realtime model, and deployment to LiveKit Cloud or self-hosted servers.

- Company: LiveKit
- Website: https://docs.livekit.io/agents/
- GitHub: https://github.com/livekit/agents
- Open source: Yes (Apache-2.0)
- Deployment: Library, Managed cloud, Self-hosted
- Languages: Python, TypeScript
- Best for: WebRTC voice agents with full code control
- Permalink: https://theaiengineering.com/platforms/voice-ai#livekit-agents

### Murf

Murf provides text-to-speech, speech-to-speech, and dubbing through APIs and SDKs, including the Falcon text-to-speech model built for voice agents, alongside voice generation tools for creators and businesses.

- Website: https://murf.ai
- Open source: No
- Deployment: Managed cloud, API
- Best for: Multilingual text-to-speech for agents and content
- Permalink: https://theaiengineering.com/platforms/voice-ai#murf-ai

### OpenAI Realtime API

The OpenAI Realtime API streams audio to and from OpenAI's realtime speech models for voice agents, connecting over WebRTC, WebSocket, or SIP, and it handles turns, tools, interruptions, and handoffs with the OpenAI Agents SDK.

- Company: OpenAI
- Website: https://developers.openai.com/api/docs/guides/realtime
- Open source: No
- Deployment: API
- Best for: Speech-to-speech agents on OpenAI models
- Permalink: https://theaiengineering.com/platforms/voice-ai#openai-realtime-api

### Pipecat

Pipecat is an open-source Python framework for real-time voice and multimodal agents, maintained by Daily, that composes speech, language, and voice services into pipelines over WebRTC, SIP, or PSTN and runs self-hosted or on Pipecat Cloud.

- Company: Daily
- Website: https://www.pipecat.ai
- GitHub: https://github.com/pipecat-ai/pipecat
- Open source: Yes (BSD-2-Clause)
- Deployment: Library, Managed cloud, Self-hosted
- Languages: Python
- Best for: Provider-agnostic voice pipelines in Python
- Permalink: https://theaiengineering.com/platforms/voice-ai#pipecat

### Retell AI

Retell AI is a platform for phone-call voice agents with a visual flow builder, testing and simulation, call analytics, SIP and existing phone number support, human handoff, and on-premises deployment for regulated industries.

- Website: https://www.retellai.com
- Open source: No
- Deployment: Managed cloud, API, Self-hosted
- Best for: Call automation built with a visual flow editor
- Permalink: https://theaiengineering.com/platforms/voice-ai#retell-ai

### Rime

Rime builds voice models for real-time conversation, with the Coda and Mist text-to-speech models for cascaded voice agents, the Prism speech-to-speech model, speech-to-text, and enterprise voice cloning.

- Website: https://www.rime.ai
- Open source: No
- Deployment: Managed cloud, API
- Best for: Conversational voices for phone and contact-center agents
- Permalink: https://theaiengineering.com/platforms/voice-ai#rime-ai

### Soniox

Soniox provides one speech AI API for real-time speech-to-text, text-to-speech, and speech translation across many languages, with low-latency streaming and support for speakers who switch languages mid-conversation.

- Website: https://soniox.com
- Open source: No
- Deployment: Managed cloud, API
- Best for: Multilingual live transcription and translation
- Permalink: https://theaiengineering.com/platforms/voice-ai#soniox

### Speechmatics

Speechmatics builds speech-to-text APIs for real-time and batch transcription across many languages, accents, and multiple speakers, aimed at regulated industries, with deployment on-device, on-premises, or in the cloud.

- Website: https://www.speechmatics.com
- Open source: No
- Deployment: Managed cloud, API, Self-hosted
- Best for: Transcription with on-premises or on-device deployment
- Permalink: https://theaiengineering.com/platforms/voice-ai#speechmatics

### Synthflow

Synthflow provides AI voice agents for contact centers, BPOs, and multi-location brands that answer and handle phone calls inside the phone systems and CRMs those businesses already run.

- Website: https://synthflow.ai
- Open source: No
- Deployment: Managed cloud
- Best for: Contact centers deploying voice agents
- Permalink: https://theaiengineering.com/platforms/voice-ai#synthflow

### TEN Framework

TEN Framework is a framework for real-time multimodal voice agents supported by Agora, with RTC and WebSocket transport and voice activity and turn detection, released under Apache-2.0 with added conditions that bar competing with Agora's offerings.

- Website: https://theten.ai
- GitHub: https://github.com/TEN-framework/ten-framework
- Open source: No
- Deployment: Library, Self-hosted
- Best for: Real-time voice agents alongside Agora services
- Permalink: https://theaiengineering.com/platforms/voice-ai#ten-framework

### Twilio ConversationRelay

Twilio ConversationRelay connects phone calls on Twilio to the AI model of your choice through a WebSocket API, handling speech-to-text, text-to-speech, and automatic language detection so applications exchange text with the call.

- Company: Twilio
- Website: https://www.twilio.com/en-us/products/conversational-ai/conversationrelay
- Open source: No
- Deployment: Managed cloud, API
- Best for: Adding an LLM to existing Twilio voice numbers
- Permalink: https://theaiengineering.com/platforms/voice-ai#twilio-conversationrelay

### Ultravox

Ultravox is a real-time voice AI platform built on speech-native models, with APIs, a console, telephony integrations, and tools for building voice agents; the company was formerly known as Fixie and is now part of Inworld.

Inworld acquired Ultravox in 2026, and ultravox.ai remains available for building voice agents.

- Company: Inworld
- Website: https://www.ultravox.ai
- Open source: No
- Deployment: Managed cloud, API
- Best for: Speech-native voice agents with telephony
- Permalink: https://theaiengineering.com/platforms/voice-ai#ultravox

### Vapi

Vapi is a developer platform for voice AI agents that make and receive phone and web calls, configured through an API, dashboard, or config-as-code, with a choice of model, transcription, and voice providers, squads of assistants, and simulations.

- Website: https://vapi.ai
- Open source: No
- Deployment: Managed cloud, API
- Best for: API-driven phone agents with swappable model providers
- Permalink: https://theaiengineering.com/platforms/voice-ai#vapi

## How to choose

1. **Pipeline type**: Cascaded pipelines chain speech-to-text, a language model, and text-to-speech for control over each stage, while speech-to-speech models reply directly in audio with fewer hops.
2. **Latency and turn-taking**: Streaming, voice activity and end-of-turn detection, and interruption handling decide whether a conversation feels natural.
3. **Languages and voices**: Supported languages, accents, language switching within a call, and voice cloning options differ between providers.
4. **Telephony and transport**: WebRTC, WebSocket, SIP, and phone number support determine whether an agent can run in a browser, an app, or a call center.
5. **Deployment and compliance**: On-premises or self-hosted models, data retention terms, and certifications such as HIPAA and SOC 2 matter for regulated calls.
6. **Control**: Code-first frameworks give full control over the pipeline, while hosted platforms trade some flexibility for dashboards, call flows, and built-in analytics.

## Frequently asked questions

### What is a voice AI agent?

A voice AI agent is software that holds a spoken conversation and takes actions for a user, such as answering a support call or booking an appointment. It either chains speech-to-text, a language model, and text-to-speech, or uses a single speech-to-speech model, and it calls tools during the conversation. Frameworks such as LiveKit Agents and Pipecat and platforms such as Vapi are used to build them.

### LiveKit vs Pipecat: what is the difference?

Both are open-source frameworks for real-time voice agents that let developers mix speech-to-text, language model, and text-to-speech providers in code. LiveKit Agents runs agents as participants in LiveKit's WebRTC rooms, offers Python and Node.js SDKs, and deploys to LiveKit Cloud or self-hosted servers. Pipecat, maintained by Daily, is a Python pipeline framework that works over WebRTC, SIP, or phone transports and runs self-hosted or on Pipecat Cloud.

### Vapi vs Retell: how do they compare?

Vapi and Retell AI are both hosted platforms for voice agents that handle phone calls. Vapi is API-first, configured through its API, dashboard, or config-as-code files, with a choice of model, transcription, and voice providers and squads of assistants. Retell AI centers on a visual flow builder, call testing and simulation, analytics, and human handoff, and it offers on-premises deployment. Both connect to telephony providers such as Twilio.

### Deepgram vs ElevenLabs: what is the difference?

Deepgram offers speech-to-text, text-to-speech, and a Voice Agent API for enterprise voice applications, with real-time streaming transcription. ElevenLabs offers text-to-speech, voice cloning, speech-to-text, sound effects, and a platform for conversational voice agents, with a large library of voices. Many teams combine providers, for example Deepgram for transcription and ElevenLabs for synthesis, through a framework such as Pipecat or LiveKit Agents.

## Related guides

- [What are AI agents? How agents work and when to use one](https://theaiengineering.com/articles/what-are-ai-agents): AI agents are LLM systems that call tools in a loop until a task is done. Learn how the loop works, agents vs agentic AI, examples, failure modes, and limits.
- [What is tool calling? How LLM function calling works](https://theaiengineering.com/articles/tool-calling): Tool calling lets an LLM request actions from your code through structured calls. Learn the request loop, parallel calls, structured outputs, security, and MCP.

Product details change quickly; check each vendor's site for current features, licensing and pricing.

## Cite this page

The AI Engineering Editors. "Voice AI." The AI Engineering. https://theaiengineering.com/platforms/voice-ai

- Title: Voice AI
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- URL: https://theaiengineering.com/platforms/voice-ai
- Markdown: https://theaiengineering.com/platforms/voice-ai.md
