Running Voice Agents Offline: A Local-First Architecture

To keep a voice agent answering when the internet drops, everything a call touches must run on site: a local PBX with a PRI or analog line, the platform, and the speech-to-text and voice models. The language model can be local too, or hosted with a local fallback. Dograh, the open-source AI voice agent platform, runs on your own server.

Key Takeaways

• A SIP trunk over the internet stops with the link, so use a PRI line.
• Speech-to-text and the voice have no fallback, so run them on site.
• A hosted language model can fall back to a local one when requests fail.

This post is part of our guide to running voice AI on local, self-hosted models, where a local stack starts with calls on servers you run. Below, we follow one call through an outage, from the phone line to the tools, and end with a test on one Dograh box.

What an internet outage takes from a voice agent

When a site's internet link fails, every part of a call that lives elsewhere stops at once. A SIP trunk from a hosted carrier reaches you over that link, so calls stop arriving. Hosted speech, language and voice models stop answering, tools that call cloud services time out, and an agent on a hosted platform is out of reach entirely.

What keeps working is whatever sits on the site itself. A phone line that doesn't use the internet, a voice agent platform on a local server, local models and the business systems on the same network can carry a call with the link down. That is the design for a rural clinic, a factory floor, a ship or a branch office whose link fails now and then and whose phones must still be answered.

The goal here is resilience. Our post on air-gapped voice agents covers a stack where nothing may leave the network, including every outbound connection a default install makes and how to switch each one off. Here the internet is allowed, and the agent simply keeps working without it.

Keep the phone line on a local PBX

The phone line is the part most local stacks miss. A call from the public phone network always comes through a carrier, and the question is whether it reaches you over the internet. A PRI line or an analog line wired into an on-site Asterisk PBX does not, so calls keep arriving when the internet link is down.

Asterisk handles those lines through DAHDI, the "Digium Asterisk Hardware Device Interface". The DAHDI README lists drivers for digital T1, E1, J1 and BRI cards and analog FXS and FXO cards. That part belongs to the PBX and its cards, so plan it with whoever runs your phones.

Dograh connects to that PBX over Asterisk ARI on your own network. The Asterisk ARI guide lists the Asterisk versions it needs. On ARI, Transfer Call can hand a caller to a staff extension on the same PBX, which also works with the link down.

Our post on connecting an AI voice agent to FreePBX walks through the setup, and our guide to open-source IVR and AI phone systems covers the wider PBX choice.

Speech and voice must already be local

The speech-to-text and voice models are the parts that can't switch over mid-outage. Dograh offers a fallback setting for the language model only, so if transcription or the voice runs on a hosted service, the call breaks when the link does. Run both on the same server as Dograh from the start.

The usual route is a Speaches server holding faster-whisper or Parakeet for speech-to-text and Kokoro or Chatterbox for the voice, connected through a Local Models (Speaches) provider. Speech-to-speech models in Dograh are all hosted, so an offline agent is a cascade of three separate models.

Pre-recorded clips help here too. The greeting, the closing line and other fixed sentences play straight from recordings stored on the server, so they need no voice model at all.

Open Source Alternative to Vapi / Retell

Self-hosted voice agent platform — no per-minute fees

dograh-hq/dograh

Star on GitHub

A hosted language model with a local fallback

The language model is the one part that can stay hosted, because Dograh can switch it over. An LLM fallback points a model configuration at a second language model, such as a 4-bit 27B model on your own GPU, and Dograh moves to it when the first one fails. With the internet down, requests to a hosted model fail or go quiet, and the local model takes the turn.

Each fallback rule has a condition. "Error" switches when the hosted request fails, and "no output" switches after a wait you set, from 100 to 10,000 ms with 1,500 ms as the default. You can add at most two rules, one of each, and the fallback target is one of your own model connections rather than a Dograh model. The LLM settings cover the model configurations the rule sits on.

Fallbacks apply only before the answer starts or a tool runs, so a reply already being spoken does not switch halfway. They are also off by default, so turn the rule on and keep the wait short, since every millisecond of it is silence on the line.

Point tools at systems on the same network

A tool that calls a system on the same network keeps working during an outage, while one that calls a hosted service fails after its timeout. Point the agent's lookups at a database, booking system or ERP on site wherever you can. An HTTP API tool waits 5 seconds by default, does not retry, and then returns the error to the agent.

For the tools that must reach the internet, plan what the caller hears. Set a tool's Messages so a line plays when the tool starts and another while it is still running, and give the agent a fallback line for a failed lookup, as our post on what a voice agent should say during an outage describes. Webhook nodes make up to 5 attempts, so after-call records catch up once the link returns.

The knowledge base needs one setting. Chunked search depends on a hosted embeddings service, so an offline agent uses Full Document mode, which hands the whole document to the language model.

Size one box and test an outage

One server with a 24 GB GPU holds the standard local stack, Qwen3.8-27B at 4-bit beside Parakeet with Kokoro for the voice, and Dograh itself needs no GPU. Self-hosted Dograh runs 10 calls at once by default, and it accepts private network addresses for the models, so nothing needs a public URL.

Calls inside the building can skip the phone line. Browser test calls and website widget calls on your own network go straight to Dograh, and the coturn server in Dograh's Docker setup handles the audio path on a closed network. Updates are the one job that needs the link, so pull new images while it is up.

Self-hosted Dograh has no per-minute platform fee, but servers, model keys and phone minutes still cost money. Dograh Cloud charges a 1¢-a-minute platform fee. An offline site runs self-hosted, so its costs are the server, the phone line and any hosted model it uses when the link is up. For the whole on-site stack, see self-hosted voice AI on open-weight models.

Then test the outage on purpose. Unplug the internet link, call in over the PRI or analog line, and run through a full conversation, a lookup and a transfer. Read each call's trace afterwards to confirm the fallback took over and every stage stayed local.