The Ultimate Guide to AI Voice Agents

An AI voice agent is software that answers or places phone calls, understands what the caller says and talks back while it reads and writes your business systems. It is worth using when the same calls arrive every day and each one ends either finished or with a person who already has the details written down.

Key Takeaways

• Start with one repeatable call the agent can finish or hand over cleanly.
• Decide where the platform and the models run before you build anything.
• Test on a real phone line, then review a sample of calls every week.

This guide helps you decide whether an AI voice agent fits. It covers how a call runs, which jobs to start with, platforms, cost and rules, and shows each part in Dograh, the open-source AI voice agent platform we build. You can self-host it or run it on Dograh Cloud.

When an AI voice agent is the right tool

An AI voice agent earns its place when it either finishes the call or passes it to a person who already has the details. Callers judge it on that and little else. In Verint's 2026 customer experience survey of 5,000 US consumers, 61% said they prefer a human agent, yet nearly 70% of that group would use AI if it resolved their issue. Two-thirds had phoned a company in the past year, so the phone still carries a large share of the work.

Most businesses are still at the start. The Census Bureau's business survey found 19.8% of US businesses using AI, rising to 37% among firms with 250 or more employees. Many teams are choosing their first call type right now, and one call type is the right place to begin.

The calls that suit an agent share a shape. They come in often and follow a pattern, and the answer sits in a system the agent can read, such as a booking calendar or an order database. Calls that need judgement or a long relationship stay with your team. For those, the agent's job is a quick handover with the caller's name and reason already written down.

How a call works, from ring to hang-up

Every voice agent call runs through the same five steps, whatever the platform. First, the call reaches the agent: a phone number at a carrier such as Twilio points at it, or the agent dials out from a list. Second, speech-to-text turns the caller's words into text as they speak.

Third, a language model reads that text with the agent's instructions and decides what to say, or which tool to run, such as a calendar lookup. Fourth, text-to-speech turns the reply into audio, or the agent plays a recorded clip. Fifth, when the call ends, the transcript, the recording and any details the agent captured go to your systems.

Each step has its own choices and failure points. Our Voice AI Explained guide follows one call in detail with a full glossary, and How the Voice AI Pipeline Works times each step from speech-to-text to the spoken reply.

One detail changes how you test. Most phone calls still reach the agent as 8 kHz audio, while a browser demo runs at 16 kHz. An agent that sounds sharp in a demo can sound flatter and mishear more on a real line. How much depends on the speech models and the phone route, so the only fair test is a real phone call.

Inbound and outbound jobs

Inbound agents answer calls people make to you, and outbound agents call people from a list your system provides. The same agent design works for both, but the setup around it changes.

Inbound jobs include answering after hours, booking appointments, giving order status and answering questions from your own documents. In Dograh, you point a phone number's Inbound workflow setting at the agent, and several numbers can point at the same one. The caller chose to call you, so the consent question is simpler, which is why most teams start inbound.

Each inbound job needs one thing behind it. Answers to common questions come from your own files: the agent's Knowledge Base takes PDF, DOCX, DOC, TXT and JSON files of up to 5 MB each. Bookings need a live calendar, which the agent reads and writes through an HTTP API tool during the call. Order status needs your order system, and Pre-Call Data Fetch can load the caller's record from it as the call connects, so the agent already knows who is calling.

Outbound jobs include reminders, confirmations, follow-ups, surveys and sales calls. A list goes into a campaign with calling hours and retries, or your own system starts each call through the API Trigger. Campaign calling hours run in one time zone, so a list that spans time zones is split into one campaign per zone. Retries are yours to set, from 0 to 10, along with the delay between them.

Outbound also brings consent rules, covered in the compliance section below. A reminder with no offer and a sales call sit under different consent rules, so decide which one you are building before you write the script.

Open Source Alternative to Vapi / Retell

Self-hosted voice agent platform — no per-minute fees

dograh-hq/dograh

Star on GitHub

A quick check before you start

Five checks tell you whether a call type is ready for an agent. First, the call type comes in often enough that answering it well frees real staff time. A call you get twice a month is not where to start.

Second, the answer lives somewhere the agent can read it, such as a calendar, an order system or a policy file. Third, there is a system to write the result into, so the booking or the message still exists after the call ends.

Fourth, there is a person to hand over to. The agent passes a caller on whenever it cannot finish, with the details written to your system first. In Dograh, the Transfer Call tool calls your team member first and connects the caller only when someone answers. On Twilio, a short spoken introduction can be played to both sides before the two people are connected.

Fifth, for outbound calls, you hold the consent each person on the list needs. If any check fails, fix that first, because the agent is only as good as the systems and people around it.

All five checks point at one test. A voice agent is worth it when it can finish the call or hand it to a person with the details already written down. If your call type passes, start with that one, choose where the platform and the models run before you build, then test on a real phone line and review a sample of calls every week.

Where the platform and the models run

Where the platform runs and where the models run are two separate choices. Making them before you build saves a rebuild later.

The platform holds your call flows, tools, phone numbers and call records. You can rent a hosted platform, where the vendor runs everything and you pay its per-minute fee. You can use an open-source platform on its maker's cloud. Or you can run that open-source platform on your own servers. Only the open-source options let you read the code and run the same platform yourself later.

The models are the speech-to-text, the language model and the voice. They can come from the platform's own bundle, from any provider through your own keys, or from open-weight models on your own servers. Full data control comes only from the last option. With your own keys, the contract is yours, but the audio still reaches the provider.

Dograh sits in the orchestration layer, where it runs the call flow, the tools and the phone connection, and you choose the speech and language models as a separate layer. It is open-source, and you can run it on your own servers or on Dograh Cloud. You can bring your own keys for any LLM, speech-to-text and voice provider, or point it at local open models.

The phone line is a third choice. Keep your numbers in your own carrier account, such as Twilio, Telnyx, Plivo or Vonage, or connect your own phone system through Asterisk ARI. Numbers you own can be pointed at a different agent or platform without asking anyone to release them.

Two more strengths matter after launch. Every call has a full trace showing what the agent heard, what the language model received and which tools ran, so you can find the layer where a call went wrong. Built-in QA reviews calls after they end and scores each step of the flow.

What an AI voice agent costs

Every minute of an AI voice agent carries three costs: the platform fee, the models and the phone line. Each one is billed separately on most platforms, so compare them line by line.

The platform fee is what you pay the platform per minute. Vapi charges $0.05 a minute for hosting, and Retell AI charges $0.055 a minute for its voice infrastructure, with models billed on top in both cases. Self-hosted Dograh has no per-minute platform fee, but servers, model keys and phone minutes still cost money. Dograh Cloud charges a 1¢-a-minute platform fee.

The models are usually the largest line, and they swing widely with the speech-to-text, language model and voice you pick. The phone line is small but never zero. Twilio charges $1.15 a month for a local US number, $0.0085 a minute for inbound calls and $0.0140 a minute for outbound calls.

Our post on what an AI voice agent costs per minute adds up one minute line by line. If you are a startup expecting growth, price the platform fee at ten times today's minutes before you choose. Our post on voice AI for startups works through that sum, along with concurrency limits and how to switch later.

Picking the voice callers hear

The voice is the part callers notice first, and you have three options. A stock voice from a text-to-speech provider is quickest. A cloned voice copies a real person who agreed to it. Recorded clips play a real person's voice for the lines that never change.

The recorded option pairs well with the cloned one. In Dograh, you upload pre-recorded audio on the Recordings page and insert a clip into a prompt by typing "@". The language model plays a clip when one fits the moment and falls back to text-to-speech when none does. Clone the same speaker for that fallback, and the whole call stays in one voice. Recorded lines also cut text-to-speech usage, since a clip plays without synthesis.

Cloning needs care. The person cloned has to agree freely, and a clone that sounds right in a browser can sound off on 8 kHz phone audio. Among open models, Chatterbox can clone from a reference clip, and Dograh runs local voices through an OpenAI-compatible endpoint. Our post on voice cloning for AI voice agents covers when a clone helps a phone agent and when it backfires.

Compliance basics

An AI voice on a phone call counts as an artificial voice in US law, so the consent rules for automated calls apply to it. The FCC's ruling on AI-generated voices brought these voices under the Telephone Consumer Protection Act (TCPA). For outbound calls, a service call such as a reminder generally needs the person's prior express consent, and a sales call generally needs their prior express written consent.

Telling callers they are speaking with an automated assistant is your choice unless a law you are under requires it. In the EU, Article 50 of the EU AI Act requires telling people when they are interacting with an AI system. If you want to tell callers, or your location requires it, write it in the Greeting Text along with the business's name. Allow Greeting Interruption is on by default on the Start Call node, so switch it off there and the full notice plays.

Recording laws differ by state and country, and some need every party's agreement, so say the call is recorded where your law requires it. Then decide where call data goes. Recordings and transcripts sit wherever the platform runs, and the audio reaches whichever company runs your models.

Our Complete Guide to Compliant AI Voice Agents follows the data through each vendor and covers payments, healthcare and law firms. Treat these points as general information, and confirm your own consent and disclosure position with counsel before your first outbound campaign.

Getting your first agent live

Pick one call type from your check and build only that. Write the greeting, the steps the call follows and the one or two tools it needs, such as a booking lookup. Our guide on how to build an AI voice agent walks through the build from the first prompt to a live number.

Test it in the browser first, then on a real phone line. Browser test calls cannot transfer a caller to a phone number, so every transfer has to be tried over a real call. Ask a few colleagues to call with real questions, and listen to every recording from that first week.

Once it is live, review calls on a schedule. A QA node scores each finished call step by step, and a sample rate below 100% picks calls at random. Reviewing a random 5 to 10% of live calls each week catches drift without reading everything. Callers often say things nobody planned for, and many teams change the flow after the first weeks live.

Measure the call type you chose, not the agent in general. Count how many calls ended finished, how many reached a person, and how many transfers went unanswered and became callback requests. If transfers often go unanswered, the gap is staffing or the timeout, which defaults to 30 seconds, and the fix is outside the prompt.

For a small business, the first call is usually the one you miss while you are busy. Our post on AI for small business phone agents lists those calls and prices one small month.

Each part of this guide above has its own deep dive:

• Voice AI for Startups: How to Launch a Phone Agent Without Lock-In
• Voice Cloning for AI Voice Agents: The Complete Guide
• AI for Small Business: Phone Agents That Answer and Call