8kHz Phone Call ASR: Latency and Accuracy Budget

Phone-call ASR forces a decision most benchmark pages skip: you cannot independently maximize latency and accuracy on 8kHz narrowband audio. The sampling rate is fixed physics, the error profile is predictable, and every millisecond you spend chasing accuracy is a millisecond subtracted from your turn budget. This guide treats 8kHz as a budget you engineer around, and pins both axes to numbers you can measure.

Why 8kHz is a fixed constraint, not a tuning knob

Telephony is sampled at 8,000Hz. By the Nyquist theorem that caps representable frequency at 4kHz: "If you sample at 8,000 hertz, the highest sound you can represent is 4,000 hertz. Everything above that is gone, not merely degraded but absent from the recording." This is not noise you can denoise away. The high band is "discard[ed] before the recognizer ever sees it," setting a permanent ceiling.

Concretely, 8kHz sampling removes about 50% of acoustic information versus wideband, and what it removes is exactly the high-frequency energy "that distinguishes similar consonants." So do not upsample to 16kHz and expect gains. Google's own guidance is explicit: capture at 16kHz or higher where you can, but "avoid re-sampling" - in telephony "the native rate is commonly 8000 Hz, which is the rate that should be sent to the service." Send native 8000Hz; resampling invents data the recognizer treats as real.

What narrowband costs in accuracy (WER)

Legacy telephony stacks were brutal: "real-world telephony WER often runs 40-50%" once you add noise and packet loss. Modern models close much of that gap, but not uniformly. A benchmark on 16,311 streaming telephony recordings at 8kHz - "with upsampling applied only when required by a provider's model" - found overall WER from 7.7% to 28.6% depending on provider. That is a nearly 4x spread on identical narrowband audio.

Errors concentrate where narrowband hurts most. The clipped high band throws away "the high-frequency energy that distinguishes consonants like s, f, and th - exactly the sounds ASR confuses most." Digits and alphanumerics (account numbers, confirmation codes) inherit this, because they lean on fricatives and short tokens with little context to rescue them. A single aggregate WER hides this: your overall number can look fine while the digit subset that actually gates your call flow is failing. Measure a digit subset separately.

The latency budget, component by component

Real-time conversational AI should target 500-800ms end-to-end for interaction to feel natural. Spend it deliberately:

• Chunking: streaming splits audio into frames "commonly 20 to 100 milliseconds each," a latency-versus-overhead trade-off - "Smaller frames mean lower latency and more messages."

• Network: cloud transmission "typically starting around 50 ms depending on server distance" and ranging to ~300ms.

• Buffering: ~250ms of audio buffering.

• Model processing: 100-300ms.

• Endpoint detection: 200-500ms - often the largest single line item.

Those add up fast, which is why median latency flatters you and p99 tells the truth. Report both. For the endpointing line specifically, see our breakdown of end-of-turn detection model latency, and for the transport layer our guide to streaming STT latency over WebSocket.

The core trade-off: you cannot max both

Everything that buys accuracy costs latency. Longer context windows and rescoring improve WER on ambiguous fricatives but delay commitment. Smaller chunks and aggressive endpointing cut latency but hand the model less acoustic context to disambiguate the consonants narrowband already blurred. On 8kHz this is sharper than on wideband because the acoustic evidence is thinner to begin with. Pick a target turn time, then spend the accuracy budget that fits - do not tune one axis blind to the other. The same latency-vs-intelligence tension governs the whole pipeline, as we discuss in speech-to-speech vs cascaded voice agent latency.

Deployment topology: on-device vs cloud

Where you run the model moves the whole budget. On-device "eliminates network delays entirely, allowing lightweight engines to achieve sub-500 ms total latency, while cloud-based systems typically experience 500-1200 ms in real-world deployments despite vendor claims of sub-300 ms." The lever most teams miss is co-location: a hosted STT API adds a round trip you can delete by running the model next to your pipeline.

We measured this directly. Time-to-first-byte "from the Deepgram API is ~250ms whereas the TTFB hosting it locally on Cerebrium is ~110ms which is a 140ms saving" - detailed in our 500ms global voice agent build. On a full pipeline we hit 600ms end-to-end first-time-to-audio with Ultravox on Cerebrium. Cold starts can erase all of it, so size that too - see serverless GPU cold starts for voice AI.

Streaming transport and capture

Keep audio and results on one persistent bidirectional connection. Streaming's "main benefit is low incremental latency. Audio and results share one open connection, avoiding a new request handshake for every utterance." Capture at native 8000Hz, stream over WebSocket, and treat chunk size as a tunable dial rather than a default. Network placement compounds here; our notes on network latency in the inference pipeline cover the transport cost in detail. If you are self-hosting the recognizer, our Faster-Whisper real-time tuning guide is the practical starting point.

A measurement plan: pin numbers to your own calls

Benchmark WER lies about your traffic because it was computed on someone else's audio. Build a small plan instead:

1. Accuracy: compute WER on a held-out set of your real 8kHz calls, overall and on a digit/alphanumeric subset. The gap between the two is your actual risk.

2. Latency: break the budget into the five components above and log each, reporting median and p99.

3. Topology: A/B hosted API vs co-located model; we measured a 140ms TTFB saving from co-location, and self-hosting the LLM has run "50% faster and still 3x cheaper" in our work.

4. Cost: tie it back to voice AI agent cost per minute so the accuracy you buy is priced.

For multilingual or accented traffic the error floor shifts again; see transcription challenges for multilingual voice agents. The discipline is the same across all of it: 8kHz is fixed, so budget latency and accuracy together and measure both on your own calls.