Speech-to-Speech vs STT + LLM + TTS: Which to Use

Introduction

Every voice AI builder eventually hits the same fork in the road.

Do you send audio straight into a model that answers back in audio? Or do you break the conversation into three steps: transcribe it, reason over it, then speak the answer?

This isn't a minor implementation detail. The architecture you pick locks in cost, latency, compliance posture, and how easily you swap models six months from now.

Vendors love to tout speech-to-speech (S2S) as the faster option, and sometimes it is. But the real gap often comes down to hundreds of milliseconds—not the dramatic leaps marketing pages suggest.

Below: how each architecture works, where each one wins, and why more teams land on hybrid setups instead of picking a side forever.

TL;DR

• Speech-to-speech cuts pipeline handoffs and preserves tone and emotion, but sacrifices text-level debugging
• Cascaded (STT + LLM + TTS) gives you audit trails, tool-calling, and independently swappable components
• Compliance needs and tool-use requirements decide the winner more often than raw speed does
• Regulated or tool-heavy workflows favor cascaded; naturalness-first, low-compliance products favor S2S
• Hybrid strategies — running both, or blending pre-recorded voice with TTS — are becoming the practical middle ground

Speech-to-Speech vs Cascaded: Quick Comparison

Factor	Speech-to-Speech	Cascaded (STT + LLM + TTS)
Latency	Single-model inference, no sequential handoffs	Sum of three stages, reduced sharply by streaming
Cost predictability	Audio-token pricing, can scale non-linearly with call length	Per-component billing, optimizable at any single stage
Debuggability	Opaque: no text checkpoint to isolate failures	Text boundary at every handoff for root-cause isolation
Compliance readiness	Needs a parallel transcription layer for audit trails	Transcripts are a natural byproduct
Customization	LLM baked into model weights; swapping means retraining	STT, LLM, TTS each swappable independently

The latency story is more nuanced than "S2S always wins." A 2026 native-voice benchmark measured OpenAI's realtime model at 0.90 seconds and Google's at 1.14 seconds under realistic conditions. Meanwhile, a separate cascaded implementation using streaming Deepgram, Qwen, and ElevenLabs components hit 755ms measured time-to-first-audio once overlapping stages were introduced.

A well-streamed cascade can beat a poorly-optimized S2S deployment, and vice versa. Benchmark your actual stack. Don't trust a universal percentage.

What Is Speech-to-Speech (S2S)?

Speech-to-speech is a single multimodal model that consumes audio tokens and emits audio tokens directly. There's no intermediate transcript required for reasoning. The model hears and responds in one pass.

That has two real advantages:

• Paralinguistic cues: Tone, hesitation, emotion, and accent survive the round trip, so conversations often feel more human.
• Lower pipeline latency: One model replaces three sequential steps, cutting cumulative delay from STT → LLM → TTS.

Dograh AI's full-stack S2S orchestration, built on Gemini Flash Live and OpenAI's GPT-Realtime-2, has roughly halved end-to-end latency for customers while improving conversational reliability.

There are two flavors worth knowing: full-duplex models that process simultaneous audio streams with no turn-taking required, and turn-based S2S models that still lean on voice-activity-detection-style endpointing to know when someone's finished speaking.

Where Speech-to-Speech Fits Best

S2S earns its keep in emotionally driven, low-compliance interactions where how something is said matters as much as the words themselves.

• Companion and consumer social apps where tone drives retention
• Coaching and language tutoring, where hesitation and pacing carry meaning
• Luxury hospitality or concierge flows built around a premium voice feel
• Mental wellness and therapy-adjacent products where emotional nuance is the product

Independent benchmarking hasn't proven a blanket engagement win for S2S over cascaded systems. Naturalness gains are use-case specific, so A/B test on your own audience before betting the product on them.

What Is Cascaded Voice AI (STT + LLM + TTS)?

Cascaded architecture chains three independent models through a text layer. Speech-to-text transcribes the caller's audio, an LLM reasons over that transcript and drafts a reply, and text-to-speech renders the reply back into spoken audio.

The structural advantages show up in production:

• Text boundaries at every stage enable content filtering, compliance checks, tool or function calling, and business-logic injection before the customer ever hears a response
• Independent components upgrade on their own — swap in a newer LLM or a better TTS voice without retraining a fused model
• Inspectable stages keep every transcript loggable and auditable for regulated workflows

Variations range from fully custom-built pipelines to bundled cascade APIs, plus hybrid setups that blend pre-recorded human voice clips with TTS fallback to cut synthesis cost while sounding more natural. More on that shortly.

Where Cascaded Fits Best

Cascaded architecture is the default choice for regulated, high-volume, or tool-heavy workflows.

• Healthcare intake and patient scheduling calls
• Financial services, insurance claims, and debt collection
• Contact centers that need audit trails for every interaction
• Legal services where strict script adherence is non-negotiable

Anywhere PHI or PII redaction, transcript logging, or exact disclosure wording matters, the text checkpoint is what makes compliance possible.

LiveKit's engineering documentation recommends cascades as the default for most production agents because stages stay swappable, inspectable, and loggable, with mature tool-calling support that native realtime models still lack.

Speech-to-Speech vs Cascaded: Which Is Better?

There's no universal winner. Weigh these five factors before you commit:

1. Latency tolerance — how many milliseconds matter to your users?
2. Compliance and audit requirements — do you need a transcript trail by law or by policy?
3. Tool-calling and RAG needs — does your agent need to hit a CRM, a booking system, or a knowledge base mid-conversation?
4. Budget predictability — can you tolerate audio-token pricing that scales with call length?
5. Brand voice and cloning needs — how important is a consistent, ownable voice identity?

Choose S2S when naturalness and emotional nuance are the product differentiator and compliance review isn't blocking launch:

• Sub-second turn-taking matters more than a full transcript trail
• The use case is conversational (support tone, sales rapport) rather than transaction-heavy
• You can accept tighter coupling to one realtime model stack

Choose cascaded when control and auditability outweigh raw voice feel:

• You need audit trails, PHI/PII redaction, or policy review on every turn
• The agent must call tools, CRMs, or RAG mid-conversation with predictable results
• You want freedom to swap STT, LLM, and TTS as better models ship

You don't have to go all-or-nothing. Hybrid setups keep cascaded control for tools, transcripts, and redaction, then use S2S or other low-latency voice paths where naturalness matters most—so you get speed without giving up the audit trail.

Real-World Example: Choosing Both Architectures with Dograh AI

Dograh AI kept running into the same wall with customers: regulated industries needed audit-friendly cascaded pipelines, while outbound sales and support teams wanted low-latency, natural-sounding calls that felt less robotic. No single-architecture platform could satisfy both needs at once.

So Dograh shipped full-stack speech-to-speech orchestration (built on Gemini Flash Live and OpenAI's GPT-Realtime-2) alongside its existing cascaded stack. Customers now pick the right architecture per use case instead of being locked into one from day one.

The trigger was outbound economics. Conversion and per-call cost pressure on outbound campaigns pushed Dograh to build a hybrid feature: real pre-recorded human voice clips with TTS fallback, in the same cloned voice, inside the cascaded stack.

The results, as reported internally:

• S2S orchestration roughly halved end-to-end latency across the stack
• The hybrid pre-recorded + TTS feature cut voice-generation costs up to 3x
• That same hybrid approach delivered 2x better conversions on outbound calling

The takeaway is that businesses don't have to commit to one architecture forever. Platforms that support both let you match architecture to use case without rebuilding your stack or locking into a single vendor.

If you're evaluating this decision, prototype both a cascaded and an S2S voice agent on the same visual workflow builder before you commit to a production stack. Dograh's builder, open source under a BSD 2-Clause license, lets you spin up a working agent in a couple of minutes and test either path with your own STT, LLM, and TTS keys.

Conclusion

S2S is winning ground for latency-sensitive, low-compliance, naturalness-first products. Cascaded still leads for regulated, tool-heavy, cost-controlled deployments. The "right" answer depends on your workload, not on which architecture is newer.

What matters is handle time, cost per call, compliance readiness, and conversion rate: the outcomes your team actually gets measured on. Platforms like Dograh AI that support both architectures remove the pressure to guess correctly on day one. You can build for today's constraints and switch as your compliance posture or product priorities change.

Frequently Asked Questions

What is the difference between TTS and STT?

STT (speech-to-text) converts spoken audio into written text; TTS (text-to-speech) does the reverse. Both are components used inside a cascaded voice AI pipeline, working on opposite ends of the conversation.

Which STT model is best?

It depends on latency, accuracy on your specific accent or domain vocabulary, and language coverage. Benchmark leading providers on your own audio rather than relying on public leaderboards alone.

Which TTS model is considered the best?

Top TTS models are judged on naturalness (MOS scores), latency to first audio, and voice cloning quality. Hybrid approaches blending pre-recorded voice with TTS fallback can outperform pure TTS on both cost and human-likeness.

Is TTS a type of AI?

Yes. Modern TTS is built on deep learning and neural network models trained to generate natural-sounding speech waveforms directly from text input.

Is speech-to-speech more expensive than cascaded voice AI?

S2S often uses audio-token pricing that scales with conversation length, while cascaded pricing is per-component and easier to optimize. Actual cost depends on call volume and average conversation length.

Can I switch from cascaded to speech-to-speech later without rebuilding my voice agent?

Migration is asymmetric. Going from cascaded to S2S is a smaller lift; going from S2S to cascaded means rebuilding turn-taking, components, and observability. Platforms that support both from the start avoid this rework.