Open-Source Voice AI Frameworks Compared: Pipecat, LiveKit, TEN

The open-source voice AI frameworks worth comparing are Pipecat, LiveKit Agents, TEN Framework and Vocode. Each gives you agent code, while phone wiring and call review stay with you. Dograh, the open-source AI voice agent platform, adds that layer on a Pipecat fork.

Key Takeaways

• Pipecat streams audio over any transport. LiveKit Agents joins a room on a media server.
• Pipecat's carrier WebSocket routes cannot transfer calls. LiveKit's native SIP can.
• Self-hosted LiveKit needs its SIP service deployed separately before phones work.

This guide compares open-source voice AI frameworks on transport, phone calls, licence and the work left to you, then covers coding agents. We show how each part works in Dograh, the open-source AI voice agent platform we build. You can self-host it or run it on Dograh Cloud.

What a voice AI framework gives you

A voice AI framework gives you the real-time loop of a voice agent as code you run yourself. Audio arrives over a transport, a speech-to-text model turns it into text, a language model writes the reply, and a voice model speaks it back. The framework handles turn-taking and interruptions between those steps, and you pick each model.

That loop is hard to build from scratch, and it is still the smaller part of a phone agent in production. Pipecat is an excellent base, and production deployment needs significant plumbing around it. A framework ships no screen where a support lead edits the greeting and no outbound dialler with calling hours. Reviewing each finished call is also yours to build.

Our guide to the best open-source voice AI platforms compares these projects on licence and on what each one ships. This guide goes one layer down. It looks at how audio travels in each framework, how a call reaches it from a phone line, how active each repository is and what you still build before the first real call.

Five projects in one table

The table compares five open-source projects on the points that decide a phone build.

The "What you still build" column is the one to read twice, because it is where the weeks go.

Pipecat: a pipeline you can point anywhere

Pipecat treats a call as a pipeline of processors that pass frames along, and the transport at each end can be swapped. Its quickstart chains the transport input, Deepgram speech-to-text, a user aggregator with Silero voice activity detection, an OpenAI model, Cartesia text-to-speech, the transport output and an assistant aggregator. The current release needs Python 3.11 or later.

The same pipeline can answer a browser over WebRTC or a phone call over a carrier WebSocket. Pipecat's runner picks the transport when the bot starts, so the agent logic does not change. Twilio streams 8 kHz mono audio, so you set the pipeline's input and output sample rates to 8000 to avoid resampling.

The phone route decides what a call can do. Pipecat's telephony guide lists WebSocket integrations for Twilio, Telnyx, Plivo and Exotel with "No advanced call center features like transfers or reconnects". To hand a caller to a person, you move to Daily PSTN or a SIP connection, which Pipecat lists for transfers, forwarding and multi-party calls.

Hosting is the last choice. Pipecat Cloud runs the bot for you, or your own servers need public HTTPS and room for many WebSockets at once. Our Pipecat tutorial takes a bot from the quickstart to a real Twilio call, step by step.

Open Source Alternative to Vapi / Retell

Self-hosted voice agent platform — no per-minute fees

dograh-hq/dograh

Star on GitHub

LiveKit Agents: an agent inside a media room

LiveKit Agents runs your agent as a participant in a room on the LiveKit media server, and phone callers join that room over SIP. In code, an Agent class holds the instructions and an AgentSession holds the speech-to-text, language and voice models plus a turn detector. On LiveKit Cloud the quickstart reaches models through LiveKit Inference model strings, so it needs no separate provider keys.

LiveKit's telephony docs describe native SIP. An inbound trunk accepts calls from your carrier, and a dispatch rule puts each caller into a room and sends your agent there. An outbound trunk plus a SIP participant dials out. It supports cold transfers over SIP REFER and agent-assisted transfers, and LiveKit Cloud sells US local and toll-free numbers.

Self-hosting changes the phone side most. LiveKit's own wording is that "the SIP service needs to be deployed separately", and its SIP signalling port and media port range must be open to the internet. Our LiveKit Agents tutorial goes from lk agent init to an inbound trunk and a dispatch rule.

LiveKit deserves credit on testing. Its Python starter ships simulations that play full conversations against the agent, plus a debugger that runs one text turn at a time. What it does not ship is a dialler for an outbound list or a screen for people who do not write Python.

TEN Framework: graphs, Agora RTC and licence terms

TEN Framework builds an agent as a graph of extensions, and its default real-time transport runs on Agora. The voice assistant example needs an Agora App ID and App Certificate next to your model keys. It runs in Docker, and TMAN Designer gives you a visual editor for the graph and each extension's settings. The repository also has a WebSocket example and a SIP call example.

The licence needs a careful read. TEN Framework's licence file is Apache 2.0 with added conditions from Agora. You may not host the framework on end-user devices such as mobile phones, or deploy it in a way that competes with Agora's offerings. Those two conditions decide whether an on-device product or a voice platform built on TEN is allowed at all, so read the file before you commit.

Vocode: an early open voice agent library

Vocode is an MIT Python library that was one of the first open voice agent projects. It covered streaming conversations, inbound and outbound phone calls, Zoom dial-in and an outbound call placed by a LangChain agent, with integrations for many speech and voice providers.

If you run Vocode in production and are weighing a move, our post on Vocode alternatives lists what a Vocode codebase needs from a replacement and weighs each option against that list.

Writing framework code with a coding agent

Pipecat and LiveKit both ship tools that help Claude Code or Codex write current framework code, and the phone wiring still falls to you. pipecat init writes an AGENTS.md and a CLAUDE.md guide for coding agents. The Pipecat Context Hub indexes Pipecat's documentation and source on your machine, about 900 MB, and serves it to the coding agent over MCP.

LiveKit installs Agent Skills with lk skills install and runs a free docs MCP server. Its lk agent debugger lets the coding agent talk to the agent it built, one text turn at a time, and read the tool calls behind each reply. Both kits produce better code. Neither one connects a carrier or tests the agent over a real phone line.

Dograh's MCP server works at a different level. The coding agent plans with you, then builds the whole call flow with its prompts and tools and publishes it, while phone numbers, secrets, file uploads and model settings stay in the Dograh app. Our comparison of Claude Code for voice agents gives one brief to all three routes and lists what each leaves you to finish.

The production layer on the same Pipecat base

Dograh runs its call pipeline on a fork of Pipecat and adds the product a team needs to run phone agents. The fork, dograh-hq/pipecat, is a GitHub fork of pipecat-ai/pipecat, pinned in as a submodule and carrying its own fixes on top. A Pipecat developer can add a new phone provider with Pipecat transports and serializers, as the custom telephony provider guide shows.

The visual workflow builder is the screen a framework leaves out. A Start Call node holds the greeting, agent nodes hold the prompts, and pathways and tools join them into a call flow that ends with an outcome. A support lead can change a greeting there without a pull request.

Phone lines connect through Twilio, Vonage, Plivo, Telnyx, Exotel, Vobiz, Cloudonix, Asterisk ARI or WhatsApp inbound calling. Transfer Call works on Twilio, Telnyx, Plivo, Cloudonix and Asterisk ARI. On a transfer, Dograh dials the destination while the caller hears hold music and connects the call once the destination answers. No answer, a busy line, a failure or a timeout reaches the agent as transfer_failed, and the call carries on.

Outbound lists run as Campaigns. You upload a CSV, set calling hours by weekday in one time zone, and set the retries (0 to 10) and the wait between them. After each call, a QA node scores every step separately with tags, a 1-10 score, sentiment and a summary, and View Trace opens the call in Langfuse with the timing of each part of every turn.

As the orchestration layer, Dograh handles the call flow, tools and telephony, and you pick the speech and language models separately. That can be any provider with your own keys or open-weight models on your own servers. Pre-recorded clips can play your most common lines, with text-to-speech as the fallback. Our Dograh vs Pipecat comparison goes through each piece one by one.

The two ways to run it differ on cost. Self-hosted Dograh has no per-minute platform fee, but servers, model keys and phone minutes still cost money. Dograh Cloud charges a 1¢-a-minute platform fee. A self-hosted install takes 10 calls at once by default, and the limit can go higher.

How to pick a framework or a platform

Pick by who will change the agent after launch and how calls will reach it. If your developers want Python control and freedom over transport, start with Pipecat and plan a Daily PSTN or SIP route for transfers. If you want native SIP and a media server you can also use for video, LiveKit Agents fits, and LiveKit Cloud saves you running the SIP service.

TEN Framework suits teams already on Agora whose product clears its licence conditions. A Vocode codebase moves to Pipecat with the fewest code changes, since both are Python pipelines. If support or operations staff will edit the agent, and you need phone lines and campaigns on day one, Dograh gives you that on the same Pipecat engine, self-hosted or on Dograh Cloud.

Each framework topic above has its own deep dive:

• Pipecat Tutorial: From First Pipeline to a Phone Call
• LiveKit Agents Tutorial: Build and Deploy a Voice Agent
• Vocode Alternatives: Open-Source Frameworks and a Full Platform
• Claude Code for Voice Agents: LiveKit vs Pipecat vs MCP