On-Prem Voice AI for Law Firms: Every Place an Intake Call Goes

On-prem voice AI for law firms runs every part of a privileged intake call on servers the firm controls: the voice agent platform, speech-to-text, the language model, text-to-speech and the storage for recordings and transcripts. The phone side can stay in-house through the firm's own phone system. Calls that carry no case facts can run hosted.

Key Takeaways

• Self-hosting the platform is half the job; the models must run in-house too.
• Live transfer to a person needs Twilio on the phone side.
• Reminder calls with no case facts do not need on-prem.

This post is part of our guide to AI voice agents for law firms. The guide's section "Where the calls run decides what privilege can survive" sets the rule: privilege has no contract like a business associate agreement, so privileged intake runs where no outside party hears it, with the platform and the models both on the firm's servers. Here is what that takes in practice with Dograh, an open-source, self-hostable voice agent platform.

Every place an intake call goes

A privileged intake call touches more systems than most firms count. The audio crosses the phone carrier, then reaches the voice platform that runs the call. From there, a speech-to-text model turns it into text for the language model, whose reply a text-to-speech model speaks. Every call leaves a recording and a transcript behind, and traces if you collect them. On-prem means each of those runs on servers the firm controls, except the carrier, covered under the phone path below.

The reason is the one courts are now testing. In United States v. Heppner, decided in February 2026, a defendant's own use of a public AI tool was found not privileged. Venable reads the ruling narrowly: "Attorney-directed use under enforceable confidentiality frameworks may present a materially different analysis." The question a court asks is who else was in the chain and what they owed. Our post on AI Voice Agents for Law Firm Intake makes the full argument; this post is about the servers.

Two layers, and why your own keys are not enough

Dograh is the orchestration layer: it runs the call flow, tools and telephony, and the speech and language models are a separate layer you choose. Running Dograh on the firm's servers keeps the call flow, tools, recordings and transcripts in-house. It does not move the models.

With your own key (BYOK) for a hosted model provider, the contract is the firm's, but the audio still reaches that provider. BYOK moves the contract, not the data. Full data sovereignty needs both halves: self-hosted Dograh and open-weight models running on the firm's own hardware.

Dograh connects to local models in two ways. For the language model, it supports OpenAI-compatible endpoints such as Ollama and vLLM, per its LLM configuration. For speech-to-text and text-to-speech, it supports Speaches, an OpenAI API-compatible server. Our guide to running voice AI on local, self-hosted models compares the model choices.

What the servers look like

A small firm can hold the whole stack on one server with one graphics card, and the sizing is known. Dograh itself runs in Docker. For a remote server, its Docker deployment guide asks for at least 8 GB of RAM and 4 vCPUs, served over HTTPS because browsers allow microphone access only on HTTPS pages. The stack runs PostgreSQL, Redis, MinIO for recordings and transcripts, the API and the UI, with nginx in front.

The models need the graphics card. The stack we recommend is Qwen3.8 27B as the language model, quantized to 4 bits, with Parakeet TDT 0.6B v2 for speech-to-text. A 4-bit 27B model at 64K context leaves about 4.2 GiB on a 24 GB card, and Parakeet needs roughly 2 GB, so both fit. For text-to-speech, Magpie TTS gives fixed voices under NVIDIA's own open model licence, which makes it open weights rather than open source. Chatterbox, under MIT, can clone a voice. Our tutorial on building a voice agent on fully local open-source models walks through that setup.

Self-hosted Dograh runs 10 concurrent calls by default, adjustable with DEFAULT_ORG_CONCURRENCY_LIMIT; the servers and phone side set the real ceiling.

Local models have costs, and they are worth stating. As one practitioner puts it in the case for local AI in Texas law firms, "Local models may not match the best commercial cloud models on every task", and "Local systems also need their own governance, security, updates, testing, and lawyer review." For an intake call that collects names and a matter type, a 27B model is enough. Test it on your own calls before you decide.

Open Source Alternative to Vapi / Retell

Self-hosted voice agent platform — no per-minute fees

dograh-hq/dograh

Star on GitHub

The phone path

The carrier still carries the audio, so the phone side is the one part a firm chooses rather than eliminates. A firm with its own phone system can connect it to Dograh through Asterisk ARI, or bring its own SIP trunk, so the call goes from the carrier straight to servers the firm runs. Asterisk ARI is an advanced setup, not on by default.

There is one trade-off. Transfer Call, the tool that hands a live call to a person, works on Twilio only. If the firm needs live transfer to a person, the phone side runs on Twilio while the agent, models, recordings and transcripts stay on the firm's servers. Without live transfer, tell the agent in the prompt to offer a callback instead and write the request to the firm's system through an HTTP API tool.

Whichever carrier you use, keep recording and transcription on your side. A carrier that only carries the call is in a different position from one that also records it or transcribes it, so leave the carrier's own recording off and let Dograh store the recording on the firm's servers.

Running offline, and what the firm now owns

Self-hosted Dograh can run with no outbound internet when it is set up that way. The models must be local, calls must come through the firm's own phone system (Asterisk ARI) or the browser, and telemetry must be off with ENABLE_TELEMETRY=false, since it is on by default in the Docker setup. Leave out the Cloudflare tunnel and Sentry. Installing and updating still need internet or a private image registry. That is "can run offline when set up this way", not offline out of the box.

On-prem also hands the firm jobs a vendor used to do. Dograh has no retention setting and deletes nothing automatically, and there is no way to delete one call's recording and transcript in the app or API. The firm deletes from its own storage under its own policy. Recording can be switched off only for the whole server, with ENABLE_CALL_RECORDING_UPLOAD, and transcripts are still saved. Transcripts keep everything said, so the agent never asks a caller to speak a password or an account number.

The firm also patches the servers and updates Dograh and the models over time. On-prem reduces who can see a call. It does not remove the firm's own duty to secure what it keeps.

When on-prem is not worth it

Not every law firm call needs the top tier, and the test is what the call carries. Texas ethics Opinion 705 asks a lawyer to be "reasonably confident that the confidential character of the information will be respected and protected by the service provider." Privileged intake, where a prospective client describes their matter and a privilege waiver cannot be undone, is where on-prem earns its cost.

Reminder calls with no case facts can run on Dograh Cloud or with your own keys: a consultation reminder or a call that names only a missing document. Firms that want on-prem with a contract can also take Dograh's Enterprise plan, which lists on-premise or private-cloud deployment with "data residency & data perimeter". Self-hosted Dograh has no per-minute platform fee (servers, model keys and phone minutes still cost); Dograh Cloud has a 1¢-a-minute platform fee. For how other vendors handle hosting, see Best AI Voice Agents for Law Firms.