Introduction
Sign up for a voice AI vendor, and you probably think you're sharing data with one company. You're not.
A single call routes through telephony, speech-to-text, an LLM, and text-to-speech providers — often four separate companies, sometimes more. Each one touches raw audio, transcripts, or extracted customer data before your caller even hangs up.
Most buyers never see this. They read "SOC 2 compliant" on a landing page and assume that covers the whole chain. It usually doesn't.
This post breaks down exactly what data leaves your infrastructure at each stage of the voice AI pipeline. It covers how that differs between closed cloud platforms and self-hosted or private-cloud models, and what to ask before you sign anything.
Key Takeaways
- A typical voice AI call passes through three to four distinct sub-processors, each a separate exposure point
- Closed platforms store recordings, transcripts, and structured outputs by default, even when you don't need them
- Closed-vendor HIPAA, GDPR, and SOC 2 often means compliance add-ons and multiple BAAs
- Self-hosted and private-cloud architectures can keep audio and transcripts fully inside your own cloud
- Evaluate which data flows are structural versus configurable before you pick a vendor
The Voice AI Pipeline: Where Your Data Actually Travels
Every cascaded voice AI system follows the same basic path: telephony → speech-to-text (STT) → large language model (LLM) → text-to-speech (TTS). Each hop is frequently handled by a different company's servers, not the platform's own infrastructure.
What Each Stage Actually Sees
- Telephony providers see the caller's phone number and raw audio stream
- STT providers receive the full audio and generate a transcript from it
- LLM providers get the entire transcript, plus your system prompt and anything injected from a CRM lookup: names, account details, case history
- TTS providers receive the text response and synthesize it into speech
A "bring your own stack" setup makes the hop count concrete. Dograh AI's documented BYO integrations span four vendor categories:
- Telephony: Twilio, Vonage, or Telnyx
- Transcription: Deepgram, Whisper, AssemblyAI, or Google STT
- Reasoning: OpenAI, Anthropic, or Azure OpenAI
- Voice output: ElevenLabs, PlayHT, or Azure TTS
That's four separate companies with visibility into pieces of a single conversation, before you count the platform orchestrating the call.
Exposure Doesn't End When the Call Does
Recordings, transcripts, structured outputs, and analytics logs are frequently stored well beyond the conversation itself, often by default. Sentiment scores, extracted PII fields, and call outcomes get written into dashboards that nobody explicitly asked for.
The Shift Toward Speech-to-Speech
Speech-to-speech (S2S) architectures collapse the STT-LLM-TTS chain into a single hop. Models like Gemini Live and OpenAI's realtime audio mode listen, reason, and speak within one session, so fewer distinct systems touch the raw audio.
Dograh AI has shipped S2S orchestration using Gemini Flash Live and GPT-Realtime-2, roughly halving end-to-end latency versus the cascaded chain. That reduction is a clear signal of where the industry is heading.
What Leaves Your Infrastructure With Closed, Cloud-Hosted Vendors
With closed, cloud-hosted platforms, call audio and transcripts get sent to and stored on the vendor's infrastructure by default. Retention varies a lot between vendors.
Retell defaults to indefinite retention: recordings and transcripts stay forever unless you manually set a shorter window (1 to 730 days). Other platforms store call artifacts extensively but never publish a numeric default retention period.
"Bring Your Own Model" Doesn't Mean Nothing Is Stored
This is the part that catches people off guard. Even when you route STT, LLM, and TTS through your own accounts, the platform still sits in the middle handling orchestration and transport. It logs, indexes, and often retains data passing through its dashboard, regardless of which model providers you selected.
Structured Outputs Pile Up Quietly
Sentiment scores, extracted PII, call outcomes, and summary fields get stored in vendor dashboards unless you explicitly turn that off. That is a second retained data surface most teams never audit, separate from the raw call recording.
Compliance Becomes a Line Item
Closed vendors commonly gate compliance behind paid add-ons:
- Enterprise-only SSO and role-based access control
- Separately purchased HIPAA or SOC 2 modules
- Organization-wide BAA execution requiring legal review
Vapi's pricing page lists HIPAA compliance at $2,000 per month and a zero-data-retention add-on at another $1,000 per month, grouped with SOC 2, PCI, and data residency under its enterprise tier.
That is recurring cost on top of usage, plus procurement delay while legal reviews the BAA chain.
You're Trusting Claims You Can't Verify
Closed-source platforms build in an audit limit you cannot remove. You can't inspect the code that handles your data. You rely on vendor claims, certifications, and trust-center disclosures rather than independently verifiable behavior.
What Stays When You Own the Infrastructure: Self-Hosted and Private Cloud Models
The self-hosted model flips the exposure equation. The entire pipeline (telephony orchestration, STT, LLM, TTS) runs inside your own cloud or VPC. No data crosses to an external party unless you explicitly configure it to.
Local Models Remove the External Call Entirely
Support for locally hosted models changes what's possible. Options like Voxtral, Whisper, Kokoro, Canary, Qwen, Chatterbox, Coqui, and Llama let a business eliminate external API calls for voice processing altogether, keeping raw audio and transcripts fully in-house.
Dograh AI is a working example of this approach. It's open-source under a BSD 2-Clause license, self-hostable via Docker, and available as a fully managed private-cloud offering. The entire voice-agent infrastructure deploys and runs inside the customer's own cloud environment rather than Dograh's.
Compliance Overhead and Code You Can Audit
When PHI or PII never leaves your infrastructure, you're not negotiating BAAs with a chain of sub-processors or waiting on a vendor's SOC 2 renewal to go live. That overhead doesn't apply, because there's no external data flow to trigger it.
Open-source architecture also means your security team can review exactly what happens to data at every stage. No black box. No vendor claims to take on faith.
The Honest Trade-Off
Self-hosting isn't free of work. You still own:
- Encryption at rest and in transit
- Access controls and audit logging
- Retention and deletion policies
- Vendor DPAs where any external component is still in use
A fully managed private-cloud deployment absorbs the operational side (orchestration, deployments, upgrades, and reliability) while your data still never leaves your own cloud account.
Compliance Overhead: What HIPAA, GDPR, and SOC 2 Actually Require
HIPAA obligations attach to specific endpoints and systems that touch PHI, not to a vendor relationship in the abstract. That's why some closed platforms restrict PHI to a single API endpoint — it limits their own compliance surface, not necessarily yours.
A documented list of "compliant" sub-processors doesn't offload your responsibility. You still have to ensure sensitive data only reaches the approved endpoints. Miss that, and the vendor's BAA doesn't save you.
HHS guidance is explicit: a cloud provider maintaining ePHI is a business associate even if the data is encrypted and the provider never holds the key. A BAA is still required, and both parties still need their own risk analysis.
GDPR follows the same pattern on controller–processor chains. Every sub-processor that touches EU personal data needs a DPA, a lawful transfer mechanism, and ongoing oversight—not a one-time checkbox on a vendor page.
SOC 2 Type II reports help clear security questionnaires, but they don't replace your own control testing when call audio and transcripts leave your boundary. The report attests to the vendor's environment, not to how data moves once it hits their stack.
Compare that to the self-hosted model. When there's no third-party data flow to begin with, there's no BAA or DPA chain to manage:
- Fewer sub-processors and contracts to renew
- Fewer places for a compliance gap to open
- Faster procurement without waiting on vendor attestations
A Vendor Evaluation Checklist: Questions to Ask Before You Sign
Before signing with any voice AI vendor, get direct answers on three fronts.
Default processing and control:
- Which providers process my audio and transcripts by default?
- Can that routing be disabled or redirected to my own storage?
Retention and data ownership: 3. Where exactly are recordings, transcripts, and structured outputs stored, and for how long? 4. Can I bring my own encryption keys or storage buckets?
Verification: 5. Is the vendor's data-handling code auditable, or does compliance rest entirely on claims and certifications I can't independently inspect?
If a vendor can't answer these clearly, that's your answer.
Frequently Asked Questions
What type of data is protected by HIPAA?
HIPAA protects Protected Health Information (PHI) — individually identifiable health data such as diagnoses, treatment records, and patient identifiers — when handled by covered entities or their business associates.
What are the 5 main components of HIPAA?
HIPAA’s core framework has five main pieces:
- Privacy Rule — use and disclosure of health information
- Security Rule — safeguards for electronic PHI
- Breach Notification Rule — required breach reporting
- Enforcement Rule — penalty procedures
- Omnibus Rule — HITECH-related updates to the rules above
Do I need a BAA with every sub-processor my voice AI vendor uses?
Yes, under most closed-vendor models. Each sub-processor handling PHI typically needs its own BAA or must appear on the vendor's approved compliant-provider list.
Can self-hosted voice AI platforms still use cloud LLMs like GPT or Gemini?
Yes. Self-hosted platforms can optionally connect to cloud LLMs, but the architecture also supports fully local models, so businesses choose based on their own data sensitivity requirements.
Does self-hosting a voice AI platform eliminate all compliance work?
No. Self-hosting removes vendor-side compliance overhead, but your business still owns securing its own infrastructure, access controls, and retention policies.
What's the difference between data encryption and data sovereignty?
Encryption protects data in transit and at rest. Sovereignty is about where that data physically resides and who can access it. A vendor can offer strong encryption while still storing your data on infrastructure you don't control.
