Nvidia Nemotron 3.5 Transcribe: What It Means for Voice Agents

Nvidia Nemotron 3.5 Transcribe is an open-weight speech model built for recordings, not live calls. It transcribes 40 languages, labels speakers, adds word timestamps, translates to and from English and handles up to an hour of audio, with commercial use allowed. For a voice agent, it fits the after-call pass on recordings kept on your own servers.

Key Takeaways

• Nemotron 3.5 Transcribe is an offline model, so run it after the call.
• Nvidia's separate Nemotron 3.5 ASR Streaming 0.6B is built for live agents.
• Self-hosted Dograh keeps recordings in-house, so the after-call pass can stay there.

A transcript written live, a fraction of a second per phrase, leaves out speaker labels and word timestamps. We look at what Nvidia released, why its speech model belongs after the call, what it adds and how to run it on self-hosted Dograh recordings. For the wider topic, see our guide to local, self-hosted models.

What Nvidia released

Nemotron 3.5 Transcribe is an open-weight model that turns recorded speech into text, and much more than plain text if you ask for it. Nvidia's model card lists 40 languages, recordings up to one hour and a context of 65,536 tokens.

The audio comes in with a short written instruction, and the instruction decides the job. One prompt asks for a plain transcript. Another asks for speaker tags such as <spk:0> and <spk:1> on each line, with word timestamps. Ask for a translation instead, and it works in both directions between English and the listed languages. The training mix also includes tagging and redacting personal details, so the model can mark where a caller read out a date of birth.

It is a mixture-of-experts model that activates about 4.3 billion parameters per step. Nvidia ships full-precision BF16 weights plus smaller FP8 and NVFP4 versions. The quick start serves it on one H100 80 GB card with vLLM, and the hardware table adds smaller options, such as FP8 on an L40S 48 GB and NVFP4 on an RTX 5090.

The licence matters as much as the model. It is released under OpenMDW-1.1, which lets you use and share the model, changed or not, free of charge, and the card says it is ready for commercial use. If you pass the weights on, a copy of the licence goes with them.

Built for recordings, not live calls

The model card settles the main question in one line: "This is an offline model. Internal audio chunking does not make it a real-time streaming ASR service." It does offline transcription of a finished recording. It does not hand back words while someone is still talking.

A voice agent on a live call needs exactly that. In our experience, a reply feels natural when the whole round trip stays under about 800 milliseconds, and speech-to-text is usually the slowest part. A model that waits for the end of the audio cannot sit in that loop, however accurate it is.

Nvidia shipped a different model for the live job. Nemotron 3.5 ASR Streaming 0.6B is a 600-million-parameter streaming model that Nvidia describes as designed for low-latency voice agent applications. It processes audio in chunks you choose, from 80 to 1,120 milliseconds, and covers 40 language-locales, 32 of which transcribe out of the box.

So the two models split the work. The small one listens during the call, and the large one reads the recording afterwards. In a Dograh agent, the live model is the speech-to-text you pick in the agent's model settings, and the recording goes to Nemotron after the call ends. Our post on live versus recorded speech-to-text explains why most voice agents need both.

Open Source Alternative to Vapi / Retell

Self-hosted voice agent platform — no per-minute fees

dograh-hq/dograh

Star on GitHub

What it adds after a call

After the call, speed stops mattering and a careful second read starts to pay off. The live model already gave you a transcript, but it had a fraction of a second per phrase. Nemotron gets the whole recording at once.

That second read pays off in review work. A QA reviewer who reads only English can get a translation of a call held in Hindi or Spanish. A dispute about what the agent promised can be checked against word timestamps, not a rough summary.

The personal-data tags help too. They can point your team at the calls where a caller read out details that need tighter handling.

Speaker labels need one caution. The card says they tell voices apart within a recording, but they do not establish who anyone is. Dograh records the caller and the agent as separate tracks, so on a two-party call you can transcribe each track on its own and already know who spoke. Speaker tags earn their keep when several voices share one track, such as a caller with a family member on speakerphone.

Treat the output like any first draft. The card lists hallucinated speech and wrong speaker labels among its known errors, especially on long recordings with overlapping voices. Keep a person correcting the transcripts that feed a decision. Our post on post-call analysis covers what to do with transcripts once you have them.

Running it on your own Dograh recordings

Self-hosted Dograh saves each call's recording and transcript in your own storage, MinIO by default or S3, so the audio is already on your servers when the call ends. The missing piece is a trigger that tells your transcription service a new recording exists.

A Webhook node at the end of the agent sends that trigger. Its payload template can carry the call ID and four links, shown below as user_recording_url for the caller's track, bot_recording_url for the agent's, recording_url for the mixed call and transcript_url for the live transcript:

{
  "call_id": "{{workflow_run_id}}",
  "caller_audio": "{{user_recording_url}}",
  "agent_audio": "{{bot_recording_url}}",
  "full_call": "{{recording_url}}",
  "live_transcript": "{{transcript_url}}"
}

What this does: When the call ends, your service receives one message with the call ID and a link to each recording. It can fetch the caller's track, send it to Nemotron on your own GPU and save the result next to the live transcript.

Fetch the files as soon as the webhook lands, because each link gives a download that works for an hour. Phone recordings usually come in at 8 kHz, and the model's recipe wants 16 kHz single-channel audio. Resample the file before you send it. That makes the format fit, though it cannot bring back what the phone line cut.

Dograh runs the call itself, meaning the flow, the tools and the phone line. Which speech and language models sit inside that call is your pick, and the pick decides where the audio travels while the caller is talking. The after-call pass above stays on your servers. For the live call to stay there too, the live speech-to-text has to run on your hardware as well, as the pillar guide sets out. A fresh self-hosted install also sends anonymous usage telemetry by default. Turning it off means setting ENABLE_TELEMETRY=false and deleting POSTHOG_API_KEY.

Should you add it now?

Add it when a plain transcript stops being enough. Teams that take calls in several languages, or need a speaker-labelled record for disputes and compliance, get the most from a second read. A team with a few hundred English calls a week may get little more than its live transcript already gives it.

Weigh the hardware honestly. The full-precision model wants an 80 GB card, and the smaller precisions trade some accuracy for memory. That is a server of its own, sitting next to the one that runs your calls.

Start small. Run a week of recordings through it, compare each result with the live transcript, and read where the two disagree. Those differences show whether the model earns its GPU on your calls. If the reason you are self-hosting is privacy, our post on why voice data needs to stay local covers the risks that make recordings worth keeping in-house.