Real-Time vs Turn-Based Voice Agents: Which Design Fits Your Product

What people actually mean by real-time and turn-based

In telecom terms, a turn-based voice agent is half-duplex. It behaves like a walkie-talkie: audio moves in one direction at a time. The agent waits for the user to stop, runs speech-to-text, sends the text to a language model, pipes the reply through text-to-speech, and plays it back. While the agent is speaking it is effectively deaf to anything except a crude interrupt signal. A real-time agent is full-duplex. The microphone is always streaming into the model, even while the agent is talking, so voice activity, backchannels like 'mm-hmm' and interruptions can be acted on without waiting for the current response to finish. MavenAGI's write-up frames it as the difference between a walkie-talkie and a normal phone call.

The practical cut is sharper than the textbook cut. Most production voice agents shipped through 2025 and into 2026 are half-duplex with barge-in bolted on: strict turns, but the user is allowed to cut the agent off mid-sentence. The newer OpenAI and Google models, plus xAI Grok Voice and Alibaba Qwen-Audio-3.1-Realtime, are genuinely full-duplex native speech-to-speech systems where one neural network takes audio in and emits audio out. Evalgent's breakdown is a reasonable primer on the distinction.

A third class muddies the terminology. Products like ElevenLabs ElevenAgents and Retell AI call themselves realtime, but their architecture is a managed cascade: streaming STT plus an LLM plus streaming TTS, orchestrated behind one WebSocket. They give you the developer experience of a realtime API without the end-to-end model. That is a legitimate design, and often the right one. It is just not what OpenAI, Google and xAI are selling.

Why your voice bot talks over people or pauses too long

Human conversation turns over fast. Linguists measure the median gap between speakers at roughly 200 ms across 10 languages, and anything noticeably longer than that reads as sluggish. Picovoice's guide on voice UX summarizes that research and the production targets most teams aim at.

The two failure modes, over-talking and over-pausing, almost always trace back to one thing: how the system decides the user is done. Classical voice activity detection is an energy gate. It watches microphone volume and fires when silence has lasted long enough, say 500 ms. Set it short and the agent cuts off anyone who says 'I need to... think about that.' Set it long and every reply feels laggy. Deepgram's own survey, reported alongside the Flux launch, found only 2 percent of voice AI developers were very satisfied with their agent conversation quality, while 61 percent reported noticeable or major flow issues.

The newer answer is to look at the words, not just the silence. OpenAI's Realtime API ships two modes. Server VAD fires on silence duration, with OpenAI's examples showing a 0.5 threshold and 500 ms silence window. Semantic VAD runs a classifier over what the user has been saying and estimates whether the thought is complete, trading some added delay for fewer false interrupts on hedges and pauses. Deepgram Flux bakes turn detection into the speech recognition model itself, lands end-of-turn detections inside 500 ms on most utterances, and reports a roughly 30 percent reduction in false interruptions against pipeline baselines. LiveKit's open-source turn detector reports an earlier release that cut unwanted interruptions by 39 percent against a VAD-only baseline.

Barge-in, the other half of the problem, is about how fast the agent can shut up once the user starts talking. The production bar is under a few hundred milliseconds from first interrupting syllable to silence from the agent. Retell AI reports sub-300 ms response gaps on barge-in across its production fleet, per their latency face-off. Half-duplex stacks typically have to detect the barge-in, cancel the LLM, flush the TTS buffer and yield the floor in a sequence that is often sitting behind network hops.

Do you actually need a realtime API?

No, if you are shipping a voicebot that answers one short question and transfers to a human. The 400 ms you save over a cascade will not change the outcome, and you will give up voice choice, debuggability and language breadth to get it.

Probably yes, if conversation flow is the product. Companion apps, language tutors, in-game characters and sales coaches where the user is going to interrupt, hedge and talk over the agent benefit most from full-duplex. Hume AI EVI 3 is tuned exactly for that: thirty expressive styles with empathic prosody modulation at 1,200 ms end-to-end, which is slow for telephony but fine for a mobile companion app.

Not yet, if you need SIP trunking with a high SLA and your buyer is an enterprise contact center. The field is still thin. OpenAI gpt-realtime-2.1 added native SIP trunking at a 99.9 percent SLA and 320 ms p95 latency, a 25 percent cut over its predecessor per the launch coverage. Microsoft Azure AI Voice Live is the other mainstream option with built-in SIP. Most other native speech-to-speech models, Gemini Live and Grok Voice included, require a third-party telephony layer. Forasoft notes SIP over the OpenAI Realtime path adds roughly 80 to 200 ms of trunk and transcode latency on top of the base model, which is worth knowing before you promise sub-second numbers to a contact center buyer.

The quiet case for turn-based is debuggability. In a cascade you have text at every stage. You can see whether STT misheard 'fifteen' as 'fifty', whether the LLM hallucinated, or whether TTS mispronounced a term, and you can swap any component independently. Cartesia's own argument for cascade is honest about this trade: native speech-to-speech is the quickest path to a demo, and cascade is the path where cascaded stages produce text artifacts you can inspect and tune without retraining anything.

What each design costs in latency and money

The 2026 latency floor for a natural-feeling voice agent is roughly 600 to 800 ms from end of user speech to first audio out. Below that, the user cannot reliably tell they are talking to a machine on timing alone. A classical cascade with streaming at every stage can land in that band once turn detection and network overhead are accounted for.

Native speech-to-speech models can beat that on paper. Google Gemini 2.0 Flash Live is published at 150 ms end-to-end. OpenAI gpt-realtime-2.1 is at 320 ms p95. Cartesia Sonic 3.6 and Deepgram Aura-2 hit sub-90 ms on the TTS layer alone, though both require you to supply the STT and LLM. Independent benchmarks from Coval, cited by Cartesia, show real-world voice-to-voice latency for several native models in the 825 to 1,780 ms range. The headline number and the production number are rarely the same.

The cost story runs the other direction. OpenAI's flagship gpt-realtime-2.1 bills about $0.06 to $0.11 per minute in typical use, per Forasoft's pricing analysis, and can climb past $0.18 on long calls when prompt caching does not absorb the context replay. Lean cascade stacks and the mini variant sit around $0.02 to $0.05. Google Gemini 3.8 Live at $0.023 per minute and Amazon Nova 2.5 Sonic at $0.015 are the native outliers on price, mainly because they do not bill for context the way OpenAI does.

Where real-time pays off and where it is wasted

Real-time pays off in three places. First, high-value conversations where the gap between 500 ms and 1,500 ms is the difference between a sale and a drop-off, which in practice means consumer sales, concierge and some outbound. Second, anything emotional: companion apps, mental health check-ins, coaching. Hume AI EVI 3 and the emotion-aware path on Alibaba Qwen-Audio-3.1-Realtime exist because prosody carries meaning text cannot. Third, gaming and interactive entertainment, where Inworld AI ships Unity and Unreal SDKs precisely because a half-second gap breaks the fourth wall.

It is wasted on structured transactional calls where the user mostly supplies account numbers and yes/no answers. The margin there is price per completed task, not conversational feel. Deepgram Voice Agent API at $0.075 per minute with a 5.26 percent STT WER, or AssemblyAI Voice Agent API with unlimited concurrency at 5.19 percent WER, both outperform native speech-to-speech on the thing that actually matters to those calls: whether the agent heard the digits right. See our full best speech-to-speech AI APIs ranking for the full scoring.

It is also weaker on anything that has to run through a legacy phone trunk at 8 kHz. Much of the prosodic nuance that native speech-to-speech models capture is flattened by the codec on the phone line, which blunts the main sensory advantage end-to-end models have over a well-tuned cascade.

The practical checklist

Three questions decide most of the picks.

• Does the caller interrupt often? If yes, pick a design with semantic or model-based turn detection: OpenAI Realtime with semantic VAD, Deepgram Flux inside your cascade, or LiveKit's open turn detector in front of any pipeline.
• Do you need SIP trunking with an enterprise SLA? If yes, your realistic native options are OpenAI gpt-realtime-2.1 and Microsoft Azure AI Voice Live. Everything else is a cascade with a telephony partner.
• Is the conversation emotional, playful or long-running? If yes, lean native speech-to-speech: Hume AI EVI 3 for empathy, ElevenLabs Eleven Realtime v1 Max for character voice consistency across 90-plus languages, xAI Grok Voice 3.0 for a flat-rate full-duplex model.