VibeVoice is Microsoft's open voice family, built for long, multi-speaker audio such as podcasts. Its streaming model, VibeVoice-Realtime-0.5B, starts speaking in about 300 ms, but Microsoft calls it research and development only, English-first and single speaker, with an audible disclaimer in every file. That rules it out for a business phone line today.
Key Takeaways
- VibeVoice-TTS-1.5B makes long podcasts, and its code left GitHub after misuse.
- VibeVoice-Realtime-0.5B is fast on paper, but Microsoft calls it research only.
- For a live business call, run an open voice whose license and card allow it.
This post is part of our guide to open-source TTS models, where VibeVoice is built for long, multi-speaker audio and its realtime model is a research release. Below, we cover what each is for, its license and status, the realtime model on a call, and what Dograh runs instead.
What VibeVoice is built for
VibeVoice is Microsoft's family of open voice models, and its headline model was made for long recordings, not short calls. VibeVoice on GitHub describes it as a family of open-source frontier voice AI models, covering both speech generation and speech recognition, built on speech tokenizers that run at a low 7.5 Hz frame rate.
VibeVoice-TTS-1.5B is the long-form model. It generates up to 90 minutes of speech in one pass with up to four speakers, mainly in English and Chinese, which makes it a podcast and audiobook tool. It is the version listeners have rated. On the Artificial Analysis Speech Arena, where people vote blind between pairs of clips, VibeVoice 1.5B ranks 89th of 103.
VibeVoice-Realtime-0.5B is the streaming model, and it is the one anyone building a voice agent looks at. It takes text as it arrives, speaks with a single voice and can run for about 10 minutes of speech. Microsoft's newer releases in the family are on the listening side, with speech recognition models that handle long recordings and streaming input.
The license and Microsoft's own limits
The license says one thing and Microsoft's own card says another, and for a business the card is the one to follow. The code and weights are MIT, which on its own would allow commercial use. But the VibeVoice-Realtime model card states: "This model is intended for research and development purposes only." It adds: "We do not recommend using VibeVoice in commercial or real-world applications without further testing and development."
The long-form model has a harder history. Microsoft removed the VibeVoice-TTS code from its GitHub repository after it "discovered instances where the tool was used in ways inconsistent with the stated intent." The 1.5B weights are still listed on Hugging Face, but the project that published them no longer ships the code to run them from its repository.
Microsoft also withheld the acoustic tokenizer, the part that would let users build a voice from their own audio, and lists voice impersonation without explicit, recorded consent as out of scope. So VibeVoice-Realtime gives you its own voices and no way to clone a real person.
Open Source Alternative to Vapi / Retell
Self-hosted voice agent platform — no per-minute fees
dograh-hq/dograh
Star on GitHub
VibeVoice-Realtime on a live call
On speed alone, VibeVoice-Realtime is in range. Microsoft gives about 300 ms to first audible speech, depending on hardware, and a natural phone turn needs speech-to-text, the language model and the voice to come back in under about 800 ms together. That figure is Microsoft's own, so you would still time it on your own GPU.
The trouble is everything else a business line needs. The card says the model is intended for English speech, with nine other languages offered as experimental and unpredictable. It does not read code, mathematical formulas or uncommon symbols, so order numbers, prices and dates must be rewritten as words before it sees them, which a phone agent's prompt should do anyway.
The deciding point is the safeguard. Microsoft embeds an audible disclaimer in every audio file it makes, plus a watermark for provenance. On a podcast that is a fair trade. On a support or sales line, a spoken notice inside every reply is not something a customer-facing team can ship, and the research-only status means nobody is promising it will change.
Join the Dograh Community
Dograh is an OSS alternative to Vapi. Join our Slack community for queries, releases, best practices & community interactions.
What to run on a phone line instead
Pick an open voice whose license and model card both allow commercial use. The seven open models in our TTS verbalization benchmark were kokoro-82m, orpheus-3b, qwen3-tts, chatterbox-turbo, chatterbox, chatterbox-multilingual and parler-tts. There, kokoro-82m read tricky text such as prices and acronyms most accurately, with a score of 0.61, and used the least compute per clip.
Kokoro is Apache-2.0 and is the default voice behind Dograh's Local Models (Speaches) option in the voice settings. Our Kokoro TTS review covers where it reads well and where it slips. When the agent must sound like a real person, Chatterbox is MIT and clones from a short clip, behind a server that returns raw 24 kHz audio.
As the orchestration layer, Dograh runs the call flow, tools and telephony, and you choose the speech and language models separately. So the voice is a setting you can change per agent, and lines that need a warm human sound every time can play as pre-recorded clips of a real person. If you would rather use a hosted voice, our comparison of the best text-to-speech APIs for voice agents covers speed, price and cloning.
Glossary
- Long-form model
- A voice model built for long recordings such as podcasts or audiobooks, often with several speakers. VibeVoice-TTS-1.5B makes up to 90 minutes with up to four speakers, a different job from a two-minute phone call.
- First audible speech
- The first sound a caller hears after the model gets the text. Microsoft gives about 300 ms for VibeVoice-Realtime, depending on hardware, which is the figure to compare with a phone turn's budget.
- Audible disclaimer
- A spoken notice Microsoft embeds in every audio file VibeVoice makes, alongside a hidden watermark. It marks the audio as generated, and a caller would hear it in each reply.
- Acoustic tokenizer
- The part of VibeVoice that turns audio into the model's internal units. Microsoft withheld it from the realtime release so users cannot build a voice from their own recordings.
