What is inside one reply
Between the moment the person stops talking and the moment the voice starts, three things happen in a chain: speech recognition turns the audio into text, the language model decides what to answer, and synthesis turns that answer into audio. Each part takes time, and the three add up.
In SARI that whole chain replies in about 600 ms, and in about 160 ms when the question is already in its answer bank and there is nothing to think about. Opening a session, up to the first audio, takes a median of 57 ms. Those are engine figures, measured at the server.
Why 600 ms and not two seconds
In a spoken conversation between people, turns change in a few hundred milliseconds. Beyond one second the caller starts wondering whether they were heard; at two seconds they repeat the sentence or hang up. That is why the figure matters in milliseconds: it is not a marketing number, it is the threshold past which the conversation stops feeling like one.
Accepting interruptions is latency too
That is called barge-in: if the person talks while the voice is answering, the voice stops and listens. It looks like a detail and it is what separates a conversation from an answering machine. Doing it on speakerphone requires echo cancellation, otherwise the voice interrupts itself when it hears itself through the microphone.
The public demo measures this live: the time it shows runs from when you stop talking until the first samples of the reply play, with recognition, model and synthesis included.
What the network adds depending on where you are
Engine figures are measured at the server. What the visitor feels adds the network round trip to the node: from Colombia to Helsinki that is a few tenths of a second more; from Europe, almost nothing. That is why the demo shows different latency depending on the country, and why a serious deployment chooses the node’s jurisdiction also for proximity to the caller.
When a provider is slow
Recognition, model and synthesis come from different providers, and any of them can have a bad moment. In SARI, if the primary takes more than 450 ms the backup steps in: no part is a single point of failure. It does not make the average reply faster; it keeps the worst case from being a silent call.