LFM3-Audio-2B on llama.cpp: GPU Memory & TTFA

What this guide can and cannot tell you

Let's be honest up front. If you are standing up LFM3-Audio-2B for the first time on llama.cpp, you want two numbers: how much GPU memory it eats, and how long until the first audio comes back. As of this writing, no trusted source publishes exact GPU-memory footprint or time-to-first-audio (TTFA) figures for LFM3-Audio-2B. We will not invent them.

What we can give you is the honest mechanism: what the llama.cpp changelog tells us actually drives memory, where TTFA accumulates in a voice pipeline, and realistic reference points from Cerebrium's own voice-AI benchmarks so you can set targets before you ship. The goal is simple - so you can size hardware and set latency targets before you ship, then measure your own deployment to confirm.

Is LFM3-Audio-2B supported in llama.cpp?

Yes. Support landed in release b11941, which added the model directly: the notes read model : add LiquidAI/LFM3-Audio-2B and LiquidAI/LFM3-Omni-2B. The same release added a text-to-speech voice selector - tts : add a voice option for LFM3-Audio and LFM3-Omni - so you can pick an output voice at inference time.

One implementation detail matters for latency planning: flash attention was disabled for the audio depthformer (mtmd : disable flash attention for the LFM3-Audio depthformer). Flash attention is one of the biggest attention-path speedups available, so its absence on the depthformer is a reason to measure TTFA empirically rather than assume parity with a text-only model of similar size.

What drives GPU memory

For a 2B-class model, the raw weight count is the floor, not the whole story. The real variable on first deployment is what you are allowed to quantize.

In b11941, llama.cpp deliberately keeps parts of the model at full precision: llama : keep the LFM3-Audio encoder convs and codec unquantized. The encoder convolutions and the codec are numerically sensitive, so quantizing them degrades audio quality - they stay in full precision and set a hard floor on footprint no matter how aggressively you shrink the language backbone.

That floor moved in release b12013, which added quant : allow quantization for LFM3-Audio codec decoder (#31386). Crucially, the decoder is quantized with a quantization-aware-trained scheme - quantize : quantize the LFM3-Audio codec decoder, it is Q4_0 QAT - meaning the weights were trained to tolerate Q4_0, so you reclaim codec-decoder memory with far less quality loss than naive post-training quantization. If you are on b11941, that decoder is full precision; on b12013+, you have the option to shrink it.

The practical takeaway: budget against usable card memory, not the sticker, and remember the KV cache grows with context length. We walk through sizing a small model to a single card in our guide on small-language-model inference cost per request on a single GPU, and the memory-vs-precision trade for 4-bit backbones in 4-bit LLM inference on H100.

What drives time-to-first-audio (and realistic expectations)

Time-to-first-audio is a pipeline property, not a single model number. For a speech-to-speech model like LFM3-Audio, TTFA is the wall-clock gap from the end of the user's speech to the first playable audio chunk - and it is the sum of transport, the model's prefill (time-to-first-token), and the codec's time to emit the first audio frame.

We have no measured LFM3-Audio-2B TTFA to quote, so anchor your targets to general voice-AI reference points from Cerebrium's own stacks (these are not LFM3 measurements):

• In our testing of an Ultravox + Cartesia pipeline on Cerebrium, we measured an end-to-end first-time-to-audio of ~600 ms. That is a useful ceiling-of-ambition for a well-tuned cascaded stack.

• A low-latency voice architecture achieved a perceived time-to-first-audio of ~1.0 s using a filler technique - down from ~5.3 s on a GPT-4o-mini baseline.

• Sub-budgets matter. Self-hosting the LLM drove time-to-first-token to ~300 ms (versus 700 ms–1.5 s for OpenAI 4o-mini), and co-locating STT cut Deepgram's TTFB from ~250 ms to ~110 ms, as documented in our 500 ms global-scale voice agent build.

Treat ~600 ms–1.0 s as the band a tuned voice system can hit; then measure where LFM3-Audio-2B actually falls, keeping in mind the disabled flash-attention path on the depthformer.

How quantization trades memory vs latency

The two levers interact. Quantizing the codec decoder to Q4_0 (b12013+) reclaims memory and can let you fit on a smaller card - but smaller cards and dequantization overhead can move TTFA in either direction, so the win is not automatic. The encoder convolutions and codec you cannot safely quantize keep both a memory floor and an audio-quality floor in place.

If cold-start spikes dominate your tail latency more than steady-state inference, that is a separate problem from footprint; we cover it in serverless GPU cold starts for voice AI. And if your real constraint is the per-minute economics of running the stack, see voice AI agent cost per minute.

First-deployment checklist

1. Pin your build. Use b11941 for base support; move to b12013+ only if you want to quantize the codec decoder.

2. Accept the unquantized floor. Encoder convs and codec stay full precision - size memory around that, not just the backbone.

3. Choose codec-decoder precision deliberately. Q4_0 QAT is available on b12013+ and is designed to hold quality.

4. Set a TTFA target, not a guess. Start from the ~600 ms–1.0 s voice-AI band above as a reference, not a promise.

5. Benchmark your own. Measure peak GPU memory (weights + KV cache at your max context) and TTFA percentiles (p50/p95) under realistic concurrency. The depthformer's disabled flash attention means your numbers, not borrowed ones, are what count.

6. Compare TTFA methodology against how we rank it in OpenAI TTS API alternatives by time-to-first-audio.

Treat this as a sizing and expectation-setting exercise: the changelog tells you what can't shrink, the Cerebrium reference points tell you what good looks like, and your own benchmark tells you the truth.