The best voice AI with QA and post-call analysis grades each call, extracts a summary and the facts you need, and tests the agent before launch. For teams that want every step of a call scored on their own servers, Dograh is the open-source option, self-hosted or on Dograh Cloud. Retell AI sells the most complete hosted QA.
Key Takeaways
- Retell AI QA scores a sampled cohort, at $0.10 a minute after 100 free minutes.
- Vapi now points new call analysis to structured outputs over its older analysis plan.
- Dograh's QA node scores each step of test and live calls, at a sample rate.
A voice agent can pass every demo and still stall on the fourth step of real calls. We set five tests first, compare four platforms led by open-source Dograh, then talk about dedicated testers, when a contact-center QA suite fits better and how to pick one for your calls.
Five tests for QA and post-call analysis
Post-call analysis and QA are two jobs. Post-call analysis pulls facts out of a finished call, such as a summary, the caller's sentiment, a booked date or whether the call reached its goal. QA grades how well the agent handled the call, so you know what to fix in the prompt or the flow.
The first test is what gets scored. A single score for a whole call tells you something went wrong, while a score for each step tells you where. The second is the summary and sentiment, and whether you can add your own fields and checks. The third is sampling, since grading every live call costs model time and grading none leaves you blind.
Take an example booking call that ends without a booking. A single score of 4 out of 10 says the call went badly. Step scores show the greeting and the name check went fine, while the date step scored 2 because the agent kept offering slots the caller had already turned down. The first tells you to worry, and the second tells you which prompt to rewrite.
The fourth test is whether the same checks run on test calls before launch, so problems surface before a caller meets them. The fifth is cost and location, meaning what QA adds to the bill and where the transcripts and scores are stored.
Built-in QA compared, platform by platform
The table covers the QA and analysis that come inside each platform, with open-source Dograh first. Dograh is open source, and every other row runs as closed code on its vendor's cloud.
| Platform | What gets scored | Summary and sentiment | Custom checks | Tests before launch | Extra cost | Code |
|---|---|---|---|---|---|---|
| Dograh | Each step of the call, whole call as fallback | For each step, with a 1-10 score and tags | Your own QA prompt | Editor test calls and Tuner simulations | Model calls only | Open source, BSD 2-Clause |
| Retell AI | Each call in a sampled cohort | Post Call Extraction | Custom fields and metrics | Simulation testing | AI QA $0.10 a minute after 100 free minutes | Closed |
| Vapi | Each call | Summary in the analysis plan or structured outputs | Structured outputs and rubrics | Testing and simulation on every plan | Part of the plan | Closed |
| ElevenLabs Agents | Each call | Sentiment and data collection | Success evaluation criteria | Simulation, next-reply and tool-call tests | Part of the plan | Closed |
Dograh: each step of a call scored and traced
Dograh's QA node runs after a call ends and reads the transcript. It splits the call by the steps in your workflow and grades each one, with tags for problems such as dead air, a looping agent or a caller who sounds frustrated, a score from 1 to 10, a sentiment and a short summary. When a call has no clear steps, it grades the whole call instead. You can replace the default instructions with your own QA prompt to check what matters to your business.

The settings decide which calls get graded, through Minimum Call Duration, Include Voicemail Calls and Sample Rate. QA runs on test calls from the editor and on live calls, and a Sample Rate below 100% picks live calls at random. Our guide to automated QA for voice agents explains why a random 5 to 10% of live calls is enough at volume. Results show on the run page under QA Results, and the Webhook node can carry them to your own systems.
For one call, View Trace opens the run in Langfuse tracing with the speech-to-text, LLM and voice time for every turn. Before launch, the Tuner integration sends finished calls to Tuner, and Tuner can call your agent with generated caller scenarios.
QA grades with your agent's own LLM or a provider you pick, such as OpenAI, Anthropic or Google Gemini, so its model calls bill like any other. Self-hosted, the transcripts and scores stay in your own database. Running Dograh yourself means no per-minute platform fee. You still pay for servers, model keys and phone minutes, and your team looks after the database and upgrades. Support comes from the open-source community, and Enterprise adds custom SLAs. If you would rather not host it, Dograh Cloud runs it for a 1 cent a minute platform fee.
Open Source Alternative to Vapi / Retell
Self-hosted voice agent platform — no per-minute fees
dograh-hq/dograh
Star on GitHub
Retell AI: AI QA and Post Call Extraction
Retell AI has the most complete hosted QA product. Retell's AI QA scores a cohort of calls you define, with trends such as average score and resolution rate, and call-level checks for hallucinations, knowledge base accuracy, overlapping speech, sentiment and tool usage, each backed by transcript evidence. It costs $0.10 a minute of analyzed call time after the first 100 minutes.
Post Call Extraction runs after each call with a summary, a success flag and user sentiment built in, plus custom fields of four types, text, selector, boolean and number. Results arrive in the dashboard, by webhook or through the Get Call API. Simulation testing before launch comes with pay as you go.
Vapi: analysis plans and structured outputs
Vapi's original analysis plan produces a summary, structured data and a success evaluation, scored with a numeric scale, pass or fail, or a checklist. Vapi's call analysis page now recommends structured outputs over the analysis plan for extracting data, summaries and evaluations. Testing and simulation come with every plan, on top of Vapi's $0.05 a minute hosting.
ElevenLabs Agents: evaluation criteria and tests
ElevenLabs Agents grades calls against success evaluation criteria you write, collects the data fields you define and records sentiment, and sends all of it out by post-call webhook. Its agent testing page offers three test types, simulation tests that play a whole conversation, next-reply tests for one turn and tool-call tests that check the agent calls the right tool.
Join the Dograh Community
Dograh is an OSS alternative to Vapi. Join our Slack community for queries, releases, best practices & community interactions.
Dedicated testers for simulated callers at volume
Built-in QA grades calls inside one platform. Dedicated testers run hundreds of simulated callers before each release, load-test the phone line and monitor agents across platforms. Tuner starts free with 300 credits, with Builder at $39 a month for about 1,000 calls, and connects to Vapi, Retell, LiveKit, Pipecat and Dograh, where the Tuner integration is a node in the workflow. Growth at $399 covers about 12,000 calls. Cekura charges $0.25 a testing minute and $0.05 a monitored call on pay as you go, with a Startup plan at $500 a month. Coval, Hamming and Roark cover the same ground, and our list of voice AI observability and testing tools sorts these by stage, with prices and what each kind of tool cannot see.
When a contact-center QA suite fits better
Contact-center QA suites such as CallMiner and Observe.AI grade people. They score human agents against scorecards, coach them and track compliance across a whole floor. Their scorecards ask whether a person showed empathy or followed the script, while an AI agent fails in other ways, such as dead air, a tool call that never ran or an answer the knowledge base never held. Everything above grades the AI agent itself. If most of your calls are still handled by people, or you need one scorecard across people and AI agents, a suite is the better fit, and our breakdown of Observe.AI pricing shows what one costs.
Picking a voice agent with built-in QA
Start with what you need to see. If you need to know which step of a call fails, with transcripts and scores on your own servers, start with Dograh. If you want a hosted QA dashboard with cohorts and transcript evidence and are happy to pay per minute, Retell AI is the most complete. Vapi and ElevenLabs Agents suit teams that mainly want summaries, extracted fields and built-in tests.
Whichever you choose, run the checks on test calls before launch, then grade a random sample of live calls each week. Add a dedicated tester when releases come often enough that simulated callers save more time than they cost.
Make the weekly review a routine. Sort the graded calls by their lowest step score, read the worst five transcripts, and open the trace for any call where the agent went quiet. Our post on debugging voice agent calls with traces shows how to read one. Fix one prompt or one tool at a time, then watch the next week's scores for that step.
Glossary
- Sample rate
- The share of live calls that QA grades, picked at random. A 5 to 10% sample keeps model costs down while still catching common problems.
- Success evaluation
- A check that marks whether a call reached its goal, such as a booking made or a question answered, against criteria you write.
- Trace
- A record of every turn in one call, with the time spent on speech-to-text, the LLM and the voice, used to find where a call went wrong.
- Simulation testing
- Testing a voice agent with simulated callers that follow scripted scenarios, before real callers reach it.

