On-Prem Voice AI for BPOs: Keep Each Client's Calls In-House

On-prem voice AI for BPOs runs the voice agent platform, the speech and language models, and the storage for recordings and transcripts on servers the BPO controls. Client call audio stays in-house only when the models run there too. The phone carrier is the one outside part, and transfers to people still work over the BPO's own phone system.

Key Takeaways

• Self-hosting the platform is half the job; the models must run in-house too.
• Each hosted model provider is generally one more processor a client approves.
• Transfers work over Asterisk ARI, so the phone side can stay in-house.

This post is part of our guide to AI voice agents for BPO and contact centers. The guide's section "On-prem voice AI: where client call data goes" starts from one line: "A client contract often says where its call data may go, and self-hosting is how a BPO keeps that promise." Here is what that takes on real servers with Dograh, an open-source, self-hostable voice agent platform.

Every place a client's call goes

A single client call touches more systems than most BPOs count, and on-prem only works if every one of them is listed. The audio crosses the phone carrier, then reaches the platform that runs the call.

From there, a speech-to-text model turns the caller's words into text for the language model, and a text-to-speech model speaks the reply. Every call leaves a recording and a transcript behind. Traces of each turn go to Langfuse if you connect it, and a QA review grades the finished call with a language model.

On-prem means each of those runs on servers the BPO controls, except the carrier, covered under the phone path below. In Dograh the QA review uses the agent's own model by default, so a program with a local model grades its calls in-house too. Our post on call QA and scoring in contact centers covers that setting. Point Langfuse at a host you run yourself, or leave tracing off.

Why client contracts decide where calls run

A BPO usually handles call data on its client's behalf, so the client's contract sets where that data may go.

Under the EU's GDPR, a BPO in that position is generally a processor for its client. Article 28(2) of the GDPR says: "The processor shall not engage another processor without prior specific or general written authorisation of the controller." Read plainly, every hosted speech or language model that hears a client's calls is generally one more processor that client has to approve. Other countries' laws and many client contracts set similar rules. This is general information, not legal advice; confirm your position with counsel.

Self-hosting shortens that list. Running open-source code on your own servers is different from building a voice platform. The code is maintained in the open, and the BPO runs it. The duty to keep client data where the contract says sits with the BPO whichever vendor it uses. Our post on data residency for voice AI turns that into a buyer's checklist.

Two layers: the platform and the models

Self-hosting Dograh keeps the call flow, recordings and transcripts on your servers. Dograh is the orchestration layer: it runs the call flow, tools and telephony, and the speech and language models are a separate layer you choose, so self-hosting it does not move the models.

With your own key (BYOK) for a hosted model provider, the contract is yours, but the audio still reaches that provider. BYOK moves the contract, not the data. The call audio stays in-house only when the speech and language models run on your own servers too, which means open-weight models on your own hardware.

For the language model, Dograh connects to OpenAI-compatible endpoints such as Ollama and vLLM, set in its LLM configuration. For speech-to-text and text-to-speech, it connects to Speaches, an OpenAI API-compatible server. The stack we recommend is Qwen3.8 27B, quantized to 4 bits, with Parakeet TDT 0.6B v2 for speech-to-text. A 4-bit 27B model at 64K context leaves about 4.2 GiB on a 24 GB card, and Parakeet needs roughly 2 GB, so both fit. For voices, Magpie TTS gives fixed voices under NVIDIA's own licence, which makes it open weights rather than open source, and Chatterbox, under MIT, can clone a voice.

Some clients accept a named provider region instead. Dograh can point Deepgram speech-to-text at its EU or Australian host, so audio is processed in that region. That is a provider region, never on-prem. Our guide to running voice AI on local, self-hosted models compares the model choices.

Open Source Alternative to Vapi / Retell

Self-hosted voice agent platform — no per-minute fees

dograh-hq/dograh

Star on GitHub

Sizing on-prem voice AI for many client programs

A BPO runs many programs on one install, so size the servers for the busiest hour across all of them. Dograh itself runs in Docker, with PostgreSQL, Redis, MinIO for recordings and transcripts, the API and the UI, and nginx in front. A remote server needs at least 8 GB of RAM and 4 vCPUs.

Many calls at once need several API workers. The scaling setup runs them behind nginx, set with FASTAPI_WORKERS. Start at one per vCPU, capped at 8 unless you have profiled your load, and budget roughly 300-500 MB of RAM for each.

A campaign can cap its own outbound calls at once, while inbound calls share the organisation's single concurrency limit (10 by default). Self-hosted, you raise it with DEFAULT_ORG_CONCURRENCY_LIMIT, and your servers, graphics cards and phone side set the real ceiling. Past the limit, a new inbound call is rejected at once, not queued, so the overflow route sits in your phone system.

The phone path and transfers

The carrier still carries the audio, so the phone side is the one part a BPO chooses rather than removes. Many BPOs already run Asterisk, and Asterisk ARI connects Dograh to it, so calls go from the carrier to servers the BPO runs.

ARI needs Asterisk 22 or later, or Asterisk 20 LTS, with the chan_websocket and res_websocket_client modules running. Setup crosses both systems. Create the ARI user in Asterisk, enter it in Dograh, then add the Stasis application name Dograh generates to your dialplan.

Transfer Call works on Twilio, Telnyx and Asterisk ARI. It passes no summary by default; on Twilio it can play a short spoken introduction to both people. Write the details to your system with an HTTP API tool before the transfer. So a fully in-house phone path keeps live transfers to the BPO's own teams. Test each one on a real phone line, since browser test calls can't transfer.

Running offline, and what the BPO now owns

Self-hosted Dograh can run with no outbound internet when it is set up that way. The models must be local, calls must come through your own phone system or the browser, and telemetry must be off. Set ENABLE_TELEMETRY=false, and remove POSTHOG_API_KEY from the api service in docker-compose.yaml (or set it to an empty value). Installing and updating still need internet or a private image registry.

On-prem also moves work to the BPO. Dograh has no retention setting and deletes nothing automatically, so you delete from your own storage under each client's policy. Recording can be switched off only for the whole server, with ENABLE_CALL_RECORDING_UPLOAD, and transcripts are still saved. Recordings and transcripts keep whatever is said, so the call never asks for a full account, card or Social Security number; use a short check such as the last four digits, and send payment links by text or email from your own system.

Because you self-host Dograh inside your existing compliance perimeter, you don't need a separate vendor review for Dograh. The speech and language model providers you choose may still need their own review, unless those models run on your servers too. Our post on passing a SOC 2 review with a self-hosted voice agent covers what auditors ask.

Dograh starts every account on its own managed models. To keep call audio away from Dograh, open Models, switch the Organization default to BYOK with your own speech and language providers (or local models), and check that no agent has its own model override.

Self-hosted Dograh has no per-minute platform fee (servers, model keys and phone minutes still cost); Dograh Cloud has a 1¢-a-minute platform fee. BPOs that want on-prem with a contract can take Dograh's Enterprise plan, which lists on-premise or private-cloud deployment with data residency, and audit logs.