Ask a captioning vendor about latency and you will get a number in the low hundreds of milliseconds. It is probably true. It is also measuring a different thing from the one you care about, which is the gap between somebody speaking and an audience reading.
The pipeline has four segments, and the quoted figure is usually one
- Speech to interim text. A recogniser emits provisional words as it hears them. This is the segment vendors quote, and it is the fastest.
- Interim to stable. Those provisional words get rewritten (several times a second) as the recogniser reconsiders. A venue screen cannot show text that rewrites itself four times a second, so it shows only words that have survived at least one more cycle. That stability costs roughly another cycle and nobody quotes it.
- Translation, if there is any. A separate call per language. Most come back fast; the tail does not, which is why a well-built system runs a second provider alongside the first rather than after it.
- Delivery. To a screen, or out to every phone in the room. On the phone path this scales with the size of the audience.
A figure that covers only segment one is not wrong. It is a component spec, like quoting a lens’s aperture as the exposure time.
Measure the thing the audience experiences
Two numbers, both taken end to end:
Untranslated: speech to screen. Say a known word at a known moment and time when it is legible on the output. For reference, measured through our own deployed relay: 718 ms median, 1,038 ms at the 95th percentile. The product target is under 1.5 s.
Translated: speech to a phone in another language. A different number, because there is a whole extra segment. Our latest measurement, from the word to the translated caption on the venue screen: about 1.7 s median, and 2.5 s in the slowest one in ten. A phone adds its own hop on top.
Use the 95th percentile, not the average. The average is dominated by the easy cases; the tail is what an audience notices, and the tail is where a fallback translator or a congested network shows up.
Two counter-intuitive results worth knowing
A relay hop can be free, or better than free. We assumed proxying audio through our own edge cost time, and measured it: 718 ms through the deployed Worker against 749 ms going straight to the provider. It was 31 ms faster, because a cloud edge has a better path to the provider than a consumer line does. Do not assume an extra hop is a cost; measure it.
Fan-out scales with the room. Delivery to phones at the 95th percentile measured 1.5 s at a thousand connections and 2.3 s at five thousand. Everything arrives at every size. It simply arrives later. Which is why the honest recommendation is five hundred to a thousand per event: not the number the system survives, the number at which it still meets its own promise.
What to ask a vendor
- Does your figure cover speech to visible on screen, or only the recogniser?
- Median or 95th percentile? If they say “average”, ask again.
- What is the translated figure, separately?
- How was it measured, and through what? A figure from a lab with a wired connection and the vendor’s own audio is not a figure from a venue.
And the thing worth checking yourself, because no number captures it: watch how text arrives. Words appearing in lumps of three or four, once a second, read as slow even when the measurement says otherwise. A line that lands all at once is a line the room cannot read at the speaker’s pace. Latency is the number; legibility is the experience, and they are not the same thing.