What is Latency? (Voice latency / turn latency)
2 min read · Bolcho
In voice AI, latency is the delay between the caller finishing speaking and the agent starting to reply. It's the single biggest factor in whether a phone conversation feels human or robotic — above about 1.2 seconds people notice the pause, and above 2 seconds they talk over the bot or hang up.
Latency explained
Latency (often measured as 'time-to-first-word') is the round-trip time for the system to hear the caller, understand them, decide what to say, and begin speaking. It's not one number but a budget made of several stages stacked together.
The goal for natural conversation is under 800 milliseconds. Most naïve stacks miss it badly because they run each stage as a separate, sequential HTTP request.
How it works
A chained stack finalizes speech-to-text, then calls the LLM, then calls text-to-speech — each a separate round-trip. A streaming pipeline overlaps them, so audio synthesis starts on the first LLM tokens instead of waiting for the whole reply.
How Bolcho handles it
Bolcho is streaming-first: a single WebSockets/WebRTC pipeline where STT, LLM and TTS overlap, plus TTS phrase caching and India-side routing.
Sub-800ms, out of the box
You don't tune streaming, endpointing or caching — they're the default, so time-to-first-word stays under 800ms while you focus on the prompt.
Why it matters for voice AI in India
For Indian calls routed to US servers, the network hops alone can add half a second. Keeping compute India-side and streaming end-to-end is what makes a Hinglish phone conversation feel real rather than laggy.
See it in a real agent
Bolcho handles this for you end-to-end. Start a voice or chat agent free.
Frequently asked questions
What is a good latency for voice AI?
Under 800ms time-to-first-word feels natural. Above ~1.2s the pause is noticeable; above 2s callers talk over the agent or hang up.
Why is my voice agent slow?
Almost always because it chains separate STT, LLM and TTS HTTP calls, so round-trips stack to 3–5 seconds per turn. Streaming so the stages overlap is the fix.
How does Bolcho stay under 800ms?
A unified WebSockets/WebRTC streaming pipeline, TTS phrase caching, good endpointing and India-side routing — all on by default.