NVIDIA Parakeet TDT 0.6B is a 600-million-parameter speech-to-text model that loads in about 2 GB, so it runs beside a 4-bit 27B language model on one 24 GB card. Version 2 is English only and version 3 covers 25 European languages. Both are CC-BY-4.0, so commercial use is allowed if you credit NVIDIA.
Key Takeaways
- Parakeet TDT 0.6B v2 needs about 2 GB, so it fits beside a 27B LLM.
- Take v2 for English calls and v3 for 25 European languages.
- CC-BY-4.0 allows commercial use, as long as you credit NVIDIA.
This post is part of our guide to open-source speech-to-text models, where Parakeet v2 is the English pick that fits beside the LLM. Below, we compare v2 and v3 and explain what streaming means for each. Then we serve Parakeet for Dograh, the open-source AI voice agent platform we build.
Why Parakeet fits beside the LLM
Parakeet's main strength on a phone agent is its size. Keeping the speech model and the language model on one card removes a network hop from every turn and one machine from the bill. That only works if the speech model is small.
A 4-bit 27B language model at a 64K context leaves about 4.2 GiB free on a 24 GB card. Parakeet TDT 0.6B v2 has 600 million parameters and needs at least 2 GB of memory to load, so it fits in that space. Voxtral Mini 3B needs about 9.5 GB in bf16, so it wants a second card. Our guide to local, self-hosted voice models covers the rest of that one-card stack.
Parakeet is also fast. The v2 card reports a throughput of about 3,380 times real time at batch size 128. That is a batch figure on a datacenter GPU, so it shows the model is efficient. The delay your caller hears is how long one sentence takes on your own card, which you measure on test calls.
v2 or v3
Pick by language first. Parakeet TDT 0.6B v2 is English only, with a 6.05% average word error rate on the Open ASR Leaderboard sets. Parakeet TDT 0.6B v3 covers 25 European languages and detects which one the caller speaks, at 6.34% on the same English sets.
The v2 card also carries the most useful phone-line test we have seen on a model card. NVIDIA ran its test sets through 8 kHz mu-law, the audio a phone line delivers, and back to 16 kHz. The average word error rate moved from 6.05% to 6.32%. With background noise at an SNR of 5, it scored 8.23%, and at an SNR of 0 it reached 11.88%.
Your own phone path may differ, so record 20 to 30 calls and test both versions on them. For v3, the card adds 11.97% on the multilingual FLEURS sets and 7.83% on MLS, plus word and segment timestamps. Both versions take 16 kHz mono audio and transcribe up to 24 minutes in one pass.
What streaming means for Parakeet
Parakeet TDT is a finished-clip model. It transcribes a stretch of audio once it ends, and v3 can stream only through a NeMo streaming script. That fits how Dograh calls a local speech model, because Dograh waits for the caller to pause and sends that finished speech segment as one request. So Parakeet's per-sentence speed is what the caller feels.
Some hosts now sell Parakeet v3 as a realtime endpoint for voice agents. That is a hosted product built around the model. The open weights you download stream only through NeMo's script.
NVIDIA's true streaming sibling is Parakeet realtime EOU 120M. It is English only and returns text without punctuation or capitals. It streams with 80 to 160 ms of latency and emits an end-of-utterance token when the caller finishes, at 160 ms in the median on its test set. It runs through NeMo's voice agent server and comes under the NVIDIA Open Model License, so call it open weights, not open source.
What works in Dograh today is the per-segment route. For the EOU model, the next step is a streaming server in front of it. For recordings after the call, NVIDIA's offline model is covered in what Nemotron 3.5 Transcribe means for voice agents.
Open Source Alternative to Vapi / Retell
Self-hosted voice agent platform — no per-minute fees
dograh-hq/dograh
Star on GitHub
Serving Parakeet so Dograh can call it
Parakeet loaded in NeMo Python has no HTTP server, so put one in front of it. The server has to answer the OpenAI-compatible /v1/audio/transcriptions route, the request shape Dograh's local speech option sends.
Then point Dograh at it. Under Providers in the sidebar, add a Local Models (Speaches) connection with your server's address. In the model configuration, type your server's model name in the Model field, since custom names are allowed. Set the Language field to a two-letter code such as en or de, and pick the connection on the STT tab. The Transcriber settings list every speech-to-text option an agent can use.
Run a test call with your setup before going live, first in the browser and then on a real phone line. Read the transcripts for names and numbers, and check each turn's speech-to-text time in the call's trace. Brand names and product words need extra care on any speech model, so put them in your test calls.
For how Parakeet compares with Whisper and Voxtral on the same route, read Whisper vs Parakeet vs Voxtral.
CC-BY-4.0 in practice
Parakeet v2 and v3 come under CC-BY-4.0, which allows commercial use and requires attribution. In practice, credit NVIDIA as the creator of the model and link the licence wherever you ship it, such as your product's notices page or your documentation.
Whisper is MIT and Voxtral is Apache 2.0, so Parakeet's credit line is the one extra step on the open shortlist. The realtime EOU model uses the NVIDIA Open Model License instead, and a NIM container from NVIDIA carries its own terms on top of the model's, so read those separately.
Join the Dograh Community
Dograh is an OSS alternative to Vapi. Join our Slack community for queries, releases, best practices & community interactions.
A Parakeet stack on one card
A fully local agent on one 24 GB card can pair Parakeet TDT 0.6B v2 with a 4-bit 27B language model such as Qwen3.8 27B, with Dograh running the call flow, tools and telephony on the same servers. The caller's audio and transcripts stay inside your network, because no hosted model hears them.
One card serves several calls at once, but each sentence waits its turn, so load-test at the number of calls you expect. A self-hosted install runs 10 concurrent calls by default, which you can raise with DEFAULT_ORG_CONCURRENCY_LIMIT. Self-hosted Dograh has no per-minute platform fee, but servers, model keys and phone minutes still cost money. Dograh Cloud charges a 1¢-a-minute platform fee.
Start with v2 on English calls and keep v3 ready for the European languages you serve. Then let your own recorded calls settle which one stays.
Glossary
- TDT (Token-and-Duration Transducer)
- The decoder design in Parakeet TDT that predicts each token together with how long it lasts, so it can skip ahead through audio. The Open ASR Leaderboard paper found TDT and CTC decoders run fastest.
- RTFx
- Inverse real-time factor: seconds of audio transcribed per second of compute. Parakeet v2's card reports about 3,380 at batch size 128, a throughput figure, not the delay one caller hears.
- End-of-utterance token
- A marker a streaming model emits when it judges the speaker has finished. Parakeet realtime EOU 120M emits one, which an agent can use to decide when to reply.
- Mu-law
- The 8 kHz phone codec NVIDIA used to test Parakeet v2 on phone-quality audio. Its average word error rate moved from 6.05% to 6.32% after the round trip.
