AI Voice Agents for Call QA and Scoring in Contact Centers: Per Client

AI voice agents for call QA and scoring in contact centers review each finished call against the client's own scorecard. The review grades every step, flags broken client rules as tags, and runs on every call while a program is new, then on a random sample. People check the flags, and the results go to each client's system.

Key Takeaways

• Write each client's rules into the QA prompt as tags.
• Score every call at launch, then a random 5-10% sample.
• Check the scores against a person's review before trusting them.

This post is part of our guide to AI voice agents for BPO and contact centers. The guide's section "Call QA and scoring for every client" sets the rule: "Each client program needs its own scorecard, and Dograh's QA node grades calls against it after they end." Here is how to build that scorecard in Dograh, an open-source, self-hostable voice agent platform, and what to do with the scores.

What the QA node grades, and when

The QA node reviews a call after it ends and never changes a call while it runs. It sits on the canvas with no connections, and it grades every eligible call of the published agent, test calls from the editor included.

The review is per step. Each step (node) of the call gets its own tags, a 1-10 score, a sentiment and a short summary, and all the tags are also listed together for the call. There is no single call score. The reviewer also gets the call's figures: its length, the number of turns, the average and longest response delay, and the language model's average time to first word.

By default, calls under 15 seconds and calls that reached voicemail are skipped. The QA node comes with ten default tags, such as DEAD_AIR, USER_FRUSTRATED and ASSISTANT_IN_LOOP. Our post on automated QA for voice agents explains each default tag and setting, so this one stays on what a contact center adds, which is a scorecard per client.

One scorecard per client program

Each client program gets its own agent, so each gets its own QA prompt with that client's rules. A bank's card line and a retailer's order line fail in different ways, and one generic review misses both.

Write each client check as a tag. Dograh reads four keys from the review: tags, overall_sentiment, call_quality_score and summary. Tags are the field listed for the whole call, so a broken client rule written as a tag shows up next to the default ones. Any extra field you invent stays only in the review's raw response.

The prompt below is for a bank's card-services program. It keeps the default context lines, adds three client checks in VERIFY_SKIPPED, DISCLOSURE_MISSED and PERSON_NOT_OFFERED, and returns the four keys Dograh reads:

You review one step of a call handled by the card-services agent
for Client A, a retail bank. Grade the agent, never the caller.

Step purpose: {{node_summary}}
Earlier in the call: {{previous_conversation_summary}}
Call metrics: {{metrics}}

Tags: use any default tag that applies, plus these client checks.
- VERIFY_SKIPPED: the agent named a balance, card or transaction
  before the caller passed the date-of-birth check
- DISCLOSURE_MISSED: the agent ordered a replacement card without
  stating the client's replacement fee
- PERSON_NOT_OFFERED: the caller asked for a person and the agent
  did not offer a transfer

Return ONLY a JSON object:
{
  "tags": [{"tag": "TAG_NAME", "reason": "evidence from the transcript"}],
  "overall_sentiment": "positive|neutral|negative",
  "call_quality_score": <1-10>,
  "summary": "1-2 sentences"
}

What this does: Every step still gets its score, sentiment and summary, because the keys match what Dograh reads. A call where the agent read a balance before the date-of-birth check carries VERIFY_SKIPPED in its tag list, with the transcript line that proves it.

A recording notice is not a QA check. Recording starts as the call connects, so the notice goes in the Greeting Text, the first thing every caller hears.

Keep each check to something the transcript can prove. "The agent verified before naming a balance" can be checked line by line. "The caller was satisfied" cannot, so that one belongs in a survey.

How many calls to score, and who grades them

The Sample Rate (%) setting decides how many calls get reviewed. It starts at 100, and below that it picks calls at random. Run every call while a program is new and for a week after any prompt change, then drop to a random 5-10% of live calls once the program is stable.

Some clients want one rule checked on every call, such as a required disclosure. A workflow can hold more than one QA node, each with its own prompt and Sample Rate. So one node can check the client's compliance rules on every call while a second grades general quality on 10%.

Use Workflow's LLM is on by default, so the review uses the same language model as the agent. That matters for client data. Dograh is the orchestration layer: it runs the call flow, tools and telephony, and the speech and language models are a separate layer you choose, so call audio and transcripts stay in-house only when those models run on your own servers too. If a self-hosted program runs a local model, the transcripts are graded on your servers as well. Turn the setting off to pick a separate provider (OpenAI, Azure OpenAI, OpenRouter, Google Gemini or Anthropic), and every reviewed transcript then goes to that provider. Our post on on-prem voice AI for BPOs covers keeping the models in-house.

Open Source Alternative to Vapi / Retell

Self-hosted voice agent platform — no per-minute fees

dograh-hq/dograh

Star on GitHub

Call QA scores are a signal: check them

A QA score tells a person which calls to read first. It never settles on its own whether a call went well, and two studies show why.

The sentiment label is the review's reading of how a step sounded, never how the customer felt. A preprint on 70,450 support conversations found that tone and customers' own 1-to-5 ratings lined up only weakly, at a 0.36 correlation. The study covers written support conversations on a fundraising platform, not calls, so read it as a warning about tone, not a figure for voice.

The grader can also be wrong. A benchmark of LLM judges for voice agents compared their scores with human ratings on telecom and retail conversations. It found that reliability depends on which metric is judged and how the review is set up, and it recommends people review the metrics that need context.

So calibrate each program before trusting it. Have a QA lead grade a batch of the same calls against the same scorecard, compare the two, and rewrite any check where they disagree. Then agree with each client who reads flagged calls and who signs off a prompt change before it goes live.

Sending results to each client

Each call's review shows on its Run Detail page, under QA Analysis. Clients usually want the results in their own system, and the Webhook node sends them after the call.

The webhook's payload can carry the review as {{annotations.qa_xxx}} fields, along with the transcript and recording links. A temporary failure, such as a timeout or a 5xx error, is retried automatically, up to 5 attempts. Point each client program's webhook at that client's own QA or CRM system.

For a flagged call, the review says what broke and the trace says why. Each call's View Trace button opens its trace in Langfuse, and our post on reading call traces to debug a voice agent walks through finding the layer that failed.

QA summaries are in English by default, which suits most client reports. For a program that takes calls in several languages, our post on multilingual support lines in BPOs covers choosing speech models per language. Self-hosted Dograh has no per-minute platform fee (servers, model keys and phone minutes still cost); Dograh Cloud has a 1¢-a-minute platform fee. Each review is a language model call, so its cost sits with whichever model grades it.