Phone Audio Quality: Why Your Voice Agent Mishears Callers

Voice agents mishear phone callers because most calls arrive as 8 kHz audio, which drops sounds above 4 kHz, and codecs and lost packets remove more. The loss depends on the speech-to-text model, so test on real calls. Dograh, the open-source AI voice agent platform, lets you swap models, and you can self-host it or run it on Dograh Cloud.

Key Takeaways

• 8 kHz phone audio loses the high sounds that separate s from f.
• The codec penalty ranges from almost nothing to severe, by speech model.
• Vonage and WhatsApp carry 16 kHz audio into a Dograh agent.

This post is part of our guide to SIP trunk providers, where clear audio is the first thing an agent needs from a trunk. Here we follow a caller's voice down the line, through band limits, codecs and lost packets, then cover the routes and Dograh settings that help speech-to-text.

What the phone line takes out of a voice

A phone call throws away part of the voice before any model hears it. Most phone calls still reach a voice agent as 8 kHz audio, which can only carry sound up to 4 kHz. Much of what separates s from f, or th from t, sits above that line, so "Smith" and "fifty" get harder to tell apart.

A browser test call does not have this problem. A web call runs at 16 kHz, the most Dograh's pipeline uses, which is why an agent that sounds sharp in the builder can stumble on a real phone. On Dograh's carrier routes, Twilio, Telnyx, Plivo, Exotel, Vobiz, Managed SIP and Asterisk ARI all deliver 8 kHz audio.

The codec carries the narrow band. Most trunks use G.711, and Twilio's trunk codec list makes its two forms, PCMU and PCMA, generally available, while Opus, G.729 and AMR-NB go through its sales team.

Mobile calls over VoLTE or 5G can carry wideband audio when both phones and carriers support it. The call falls back to 8 kHz when the signal drops or it reaches a landline, and the leg into the agent's carrier is usually narrowband anyway.

Codecs and transcoding chains

The band limit is only the first loss. A call that crosses networks is often converted from one codec to another, and each conversion takes a little more. Caller Digital's benchmark, from a voice AI company in India, played 900 recordings from 45 speakers in five Indian languages through seven codec paths into the same speech recogniser.

Word error rate rose from 6.1% on wideband audio to 7.4% on 16 kHz Opus, 11.8% on G.711 and 15.2% on G.729. When G.711 audio was converted again to a mobile codec, AMR-NB, it reached 19.7%. That is the chain many real calls take, from a mobile network through a carrier into a trunk.

An independent test points the other way for some models. The WildASR benchmark, from researchers studying speech recognition for voice agents, simulated G.711 on clean English speech. Whisper Large V3 went from 4.2% to 4.8% and GPT-4o Transcribe from 2.8% to 2.9%, while Qwen2-Audio jumped from 5.8% to 48.3%.

Both results fit what we see on calls. In our experience, speech-to-text errors on a phone call can be up to three times higher than on a web call, but the gap depends on the model and the phone path. One model barely notices the line. Another falls apart on it. You only know which you have by testing on your own calls.

Open Source Alternative to Vapi / Retell

Self-hosted voice agent platform — no per-minute fees

dograh-hq/dograh

Star on GitHub

Packet loss and jitter

Audio that survives the codec can still be damaged on the network. A lost packet removes a slice of sound, often part of a word, and jitter, packets arriving at uneven times, makes the receiver stretch or drop audio to keep up. Speech-to-text then sees a word with a hole in it.

The fixes sit on the network side. Run the agent's servers on a stable wired connection, and pick a carrier with servers near yours. If you bring your own trunk through Dograh's Managed SIP, choose the Global, India or UAE region closest to your carrier, so the audio travels a shorter path.

Routes that carry 16 kHz audio

Two routes into Dograh skip the narrow band on the agent's side. Vonage streams 16 kHz audio, and Vonage's WebSocket audio guide calls that rate "recommended for speech recognition". WhatsApp voice calls also arrive at 16 kHz.

There is a trade-off. Transfer Call runs on Twilio, Telnyx, Plivo, Managed SIP and Asterisk ARI, so a 16 kHz route suits agents that finish calls themselves, such as surveys or order updates, while lines that hand callers to staff stay on a route that transfers. The caller's own leg can still be narrowband before it reaches Vonage. Our Vonage setup guide walks through connecting it.

What helps inside the agent

The biggest lever is the speech-to-text model, and the only fair test is your own callers on your own phone path. Place calls from a mobile and a landline, in a quiet room and a noisy one, and compare what two or three models hear. Our post on why demos sound better than real calls covers the wider gap.

You can bring your own key for any speech-to-text provider, or run an open model on your own servers, so switching after a test is a settings change. Swapping models changes the model bill, not the platform line. Self-hosted Dograh has no per-minute platform fee, but servers, model keys and phone minutes still cost money. Dograh Cloud charges a 1¢-a-minute platform fee.

Then tune what the model hears. Add product names and hard terms to the Dictionary in the agent's Settings, and use Language Hints with Dograh's own models or Deepgram Flux. Each call's trace shows exactly what speech-to-text heard on every turn, and QA on a sample of live calls flags the ones that went wrong. For noisy lines, see our post on accents and background noise.

The agent's voice suffers too. Caller Digital's panel rated a premium Hindi voice 4.4 out of 5 on wideband and 3.6 on G.711, so listen to your chosen voice over a real phone call before you launch. For details that must be exact, have the agent read them back, as our post on names, numbers and emails explains.