The best multilingual voice AI lists each language for speech-to-text and for voices, detects the caller's language and asks when it is unsure. For teams serving Indian languages or many markets on their own servers, Dograh is the open-source option, with Sarvam, Soniox or Deepgram picked per agent, self-hosted or on Dograh Cloud.
Key Takeaways
- Speech-to-text languages and voice languages are separate lists, so check both for each market.
- Retell AI now flags its ten-language Multilingual setting as legacy and favors chosen languages.
- Dograh detects the caller's language through Soniox, Deepgram or Sarvam, chosen per agent.
A caller who switches from English to Hindi halfway through a sentence tests a voice agent harder than any demo. We set five tests first, compare six platforms led by open-source Dograh, then talk about why the longest language list misleads and how to pick a platform for your callers.
Five tests for a multilingual voice agent
The first test is the speech-to-text list. The languages a platform can hear and the languages its voices can speak are two different lists, and a language belongs on your shortlist only when the vendor names it on both. The second is detection, meaning whether the agent works out the caller's language on its own or needs it set before the call.

The third test is mixed speech. Many callers mix two languages in one sentence, and only some speech models are built for that. The fourth is accents and regional variants, such as Indian English next to American English, or Mexican Spanish next to Spanish from Spain. The fifth is what happens when the agent is unsure. A good agent never carries on confidently in a language it identified with low confidence, and asks a short question first.
One rule holds everywhere. A voice speaks the language of the text it is given, so mixed-language audio from the voice is never promised. The reply language comes from the language model, and the voice has to cover it.
Multilingual support compared, platform by platform
The table takes each platform's own language page, with open-source Dograh first. Apart from Dograh, each row is closed code hosted by its vendor.
| Platform | Languages listed | How detection works | Voices | Code |
|---|---|---|---|---|
| Dograh | Set by the speech-to-text you pick per agent. Sarvam saaras:v3 covers 23, Soniox 60+, Deepgram's multilingual set 10 | Auto-detect on Soniox, Deepgram or Sarvam, or a per-language agent through Transfer To Agent | Per voice provider, such as Sarvam Bulbul in 11 Indian language codes | Open source, BSD 2-Clause |
| ElevenLabs Agents | 31 with all additional languages switched on | A language detection tool switches the language | One voice per language suggested, added languages use the v2.5 Multilingual model | Closed |
| PolyAI | Replies in 73 language codes, its Raven 3.5 model in 24 | Speech-to-text routed per language across providers, with fallback | Varies by provider and language | Closed |
| Vapi | Depends on the transcriber | Deepgram Nova 3 Multi or AssemblyAI with auto-detect, while Azure, Whisper and Speechmatics are single-language | Azure multilingual-auto or Vapi Voices V2 | Closed |
| Retell AI | One or several chosen languages per agent, plus a legacy ten-language Multilingual setting | Among the languages you pick, or set per call | Per voice | Closed |
| Synthflow | About 33, plus a Multilingual option across ten | The Multilingual option switches among its ten | American, British and Australian English accents | Closed |
Dograh: a speech model picked for each language
Dograh lets you choose the speech-to-text, LLM and voice for each agent, and the speech-to-text choice decides how languages are heard. In the speech-to-text settings, Soniox with its multi option detects the language across the 60+ on Soniox's language list, and Deepgram's Multilingual (Auto-detect) option covers its ten-language set. For Indian languages, pick Sarvam's saaras:v3 model with the language left on auto-detect. It covers 23 languages, 22 Indian plus English, and handles code-mixed speech such as Hindi sentences full of English words.
The voice comes next. Sarvam's Bulbul voices cover 11 Indian language codes, and our guide to multilingual TTS for voice agents compares voice coverage by provider. The Greeting Text plays before any detection happens, so write it in both languages when callers may open in either. Language hints, on Dograh's own models and Deepgram Flux, steer detection toward the languages you expect, and Azure Speech takes any regional code you type, such as en-IN.
When one agent cannot serve every language well, the Transfer To Agent tool hands the live call to an agent built for that language, with its own model and voice, and the transcript carries over. The new agent's language reaches the live speech-to-text stream when both agents use the same provider and model, so write the handover line in that language. When the agent is unsure which language it heard, it asks a short question before going on. If no agent fits, Transfer Call on Twilio, Telnyx, Plivo, Cloudonix or Asterisk ARI rings a person, and Dograh calls the destination first and connects the caller only when someone answers.
Self-hosted, the audio of every language stays on your servers apart from the providers you choose to call, and our Sarvam guide covers those model prices. Running Dograh yourself means no per-minute platform fee. You still pay for servers, model keys and phone minutes, and your team looks after the database and upgrades. Support comes from the open-source community, and Enterprise adds custom SLAs. If you would rather not host it, Dograh Cloud runs it for a 1 cent a minute platform fee.
Open Source Alternative to Vapi / Retell
Self-hosted voice agent platform — no per-minute fees
dograh-hq/dograh
Star on GitHub
ElevenLabs Agents: 31 languages and a detection tool
ElevenLabs Agents sets up 31 languages when you add all the additional languages, and its language settings suggest a separate voice for each. Adding languages moves the agent to the v2.5 Multilingual model, with English able to keep its own. A language detection tool switches the conversation when the caller changes language, and the first message can be written per language. On the web widget, the caller picks a language at the start and it stays fixed for that call.
PolyAI: 73 reply languages and routed speech
PolyAI separates the layers more plainly than most. Its language coverage page lists replies in 73 BCP 47 codes, with its own Raven 3.5 model covering 24 of them and the rest needing a third-party LLM. Speech-to-text is provider-dependent, routed per language across Deepgram, Google, Amazon Transcribe and others with fallback, and voices vary by provider and language. PolyAI prices its managed agents through its sales team, per minute.
Join the Dograh Community
Dograh is an OSS alternative to Vapi. Join our Slack community for queries, releases, best practices & community interactions.
Vapi: detection comes from the transcriber
On Vapi, the transcriber you pick decides everything. Vapi's multilingual guide recommends Deepgram Nova 3 with Multi, and AssemblyAI's universal-3-6-pro model adds automatic detection and code-switching. Azure, OpenAI Whisper and Speechmatics are single-language only there. For the voice, Azure's multilingual-auto or Vapi Voices V2 follow the reply language, and the guide says you must list the supported languages in the system prompt.
Retell AI and Synthflow: pick languages or a set of ten
Both started with a mode that switches among the same ten languages, and Retell AI has since moved past it. Retell's multilingual page now lets you pick one or several specific languages per agent, notes some accuracy loss when they span language families, and flags the old ten-language Multilingual setting as legacy. It also warns against relying on auto-detect for close languages such as Cantonese and Mandarin, and suggests one agent per language, routed by the dialed number or set per call through the inbound call webhook.
Synthflow lists about 33 languages, and its Multilingual option switches among English, Spanish, French, German, Hindi, Russian, Portuguese, Japanese, Italian and Dutch. English comes with American, British or Australian accents, and Synthflow recommends its own speech-to-text for non-English agents.
Why the longest language list misleads
A language on a list means a model was trained on it. How well it understands your callers on a phone line is a separate question, because phone audio is narrowband, accents vary by region, and a model can score well on clean recordings and stumble on a noisy call from a market street. PolyAI's 73 codes, for example, are reply languages, while its speech recognition depends on the provider behind each one.
Accents need the same check. Synthflow lets you choose American, British or Australian English, Azure Speech in Dograh takes regional codes such as en-IN, and Sarvam built saaras:v3 for 8 kHz phone audio from Indian callers. None of that replaces hearing your own callers.
So test each language as its own leg, with real recordings from the regions you serve, before you count it as supported. Our post on multilingual voice agents covers that test plan and what the agent should do when it is unsure.
Picking a multilingual voice agent
Start with the languages your callers actually use. If they speak Indian languages, often mixed with English, start with Dograh and Sarvam's saaras:v3, self-hosted or in the cloud. If they spread across Europe and Asia, Soniox on Dograh, ElevenLabs Agents and PolyAI cover the widest sets. If you need a hosted builder for a few chosen languages, Retell AI and Synthflow handle that well.
Whichever you choose, call it in each language from a real phone before launch, and check the greeting, a mid-call switch and the unsure moment. Our guide to language switching in voice agents shows how a mid-call switch works step by step.
Glossary
- Code-mixed speech
- Speech that mixes two languages in one sentence, such as Hindi with English words. Only some speech-to-text models, such as Sarvam's saaras:v3, are built for it.
- Code-switching
- A caller changing language between or within sentences. A model with code-switching support keeps transcribing correctly when that happens.
- Language hints
- A short list of the languages you expect, given to the speech-to-text model so detection leans toward them.
- BCP 47 code
- The standard tag for a language and region, such as hi-IN for Hindi in India or es-MX for Mexican Spanish.

