Teams building voice agents increasingly face a choice: let the speech-to-text provider handle turn detection, or keep that logic in-house. For developers who have already tuned their own voice-activity detection (VAD) over thousands of calls, the provider's built-in endpointing can feel like an unnecessary middleman. A new pattern pairs custom VAD with synchronous HTTP ASR, cutting transcription latency to roughly 134 milliseconds per clip at a cost of $0.45 per hour, giving teams full control over the conversation loop without sacrificing accuracy.
The Architectural Decision: Who Owns the Turn Boundary?
In a typical streaming setup, the STT model continuously watches audio to decide when the user has finished speaking, inserting its own judgment between 'user stopped talking' and 'you get the transcript.' For teams that already know when the turn is over—because their VAD just fired—every millisecond the model spends re-deciding is pure latency. The core question becomes: who owns the turn boundary? If the answer is the developer, the ideal STT layer does exactly one thing: turn a finished clip into accurate text, fast.
Built-in turn detection, as found in AssemblyAI's Universal-3.5 Pro Realtime, takes about 300ms to detect end-of-turn and commit the transcript. It's great for teams that want conversation-aware endpointing handled for them and value partial transcripts as the user speaks. But for those who prefer control, the sync HTTP API returns a transcript in a single response at roughly 134ms p50 for clips up to two minutes, with no end-of-turn step on the model's side. The tradeoff is clear: you give up mid-utterance streaming but gain a tighter, developer-controlled loop.
Staying Under the 500ms Budget
Responsive voice agents often target under 500ms from end-of-speech to the start of the agent's reasoning. When you own turn detection and use sync ASR, the budget breaks into three parts: your VAD's end-of-turn decision (the biggest lever you control), sync transcription at ~134ms p50, and network overhead of tens of milliseconds. Because the model isn't detecting end-of-turn, the transcription step is close to pure processing time. Keeping the HTTP connection warm between turns—reusing a single session to avoid DNS, TCP, and TLS setup—protects the latency you've worked to win.
Accuracy and the Right Tradeoff
Owning turn detection doesn't cost transcript quality. The Sync API returns the same accuracy as batch transcription—the same handling of names, numbers, and domain terms—because it runs on the same Universal-3.5 Pro model. In a voice agent, transcript accuracy is paramount: if the transcript is wrong, the LLM answers the wrong question, and no amount of speed rescues the interaction. The tradeoff is clean: you lose the model's built-in endpointing and mid-utterance streaming, but you keep full control of the conversation loop and full accuracy on the words.
When to Hand Turn Detection Back
Bring-your-own turn detection is the right pattern for teams that have genuinely tuned their VAD over real calls and want the STT layer to stay out of conversation logic. It's the wrong pattern if you're using your own VAD mainly to avoid learning someone else's—in that case, you're taking on the hardest part of voice UX to save a config page. For teams that prefer not to own endpointing, interruption handling, and orchestration, AssemblyAI's Voice Agent API builds conversation-aware turn detection and barge-in directly into Universal-3.5 Pro Realtime over one WebSocket at a flat $4.50 per hour. Both options run on the same speech accuracy underneath; the difference is how much of the loop you want to hold.