The Active-Parameter Lie: What a Mixture-of-Experts Model Actually Costs on a Serverless GPU

The number every MoE benchmark quotes vs the number your invoice charges

Every Mixture-of-Experts (MoE) announcement leads with the flattering number: the active parameter count. Mixtral 8x7B is sold on "13B active." DeepSeek-V3 is sold on "37B activated." Those figures describe compute per token, and they are real. They are also not the number your finance team pays.

On a serverless GPU, you are billed for the hardware that stays resident while your model is loaded, and every expert has to be resident whether or not a given token touches it. The active-parameter figure predicts your throughput. The total-parameter figure predicts your VRAM footprint, your GPU tier, and therefore your bill. Conflating the two is the active-parameter lie, and it is the single most common budgeting error we see teams make when they move an MoE from a benchmark PDF to production.

Active vs total parameters

An MoE layer replaces one dense feed-forward network with many "experts" and a router that picks a subset per token. In Mixtral 8x7B, "each token has access to 47B parameters, but only uses 13B active parameters during inference." A router network selects two experts per token per layer, and critically "the selected experts can be different at each timestep" - so over a single sequence, essentially every expert gets exercised.

DeepSeek-V3 makes the gap far more dramatic: it is "a strong Mixture-of-Experts (MoE) language model with 671B total parameters with 37B activated for each token." That is a vast spread between the number that sets your compute cost and the number that sets your memory cost. If you size hardware off 37B, you will not be able to load the model at all.

Why MoE demands high VRAM: all experts loaded in memory

Here is the mechanism behind the invoice. Because any token can be routed to any expert, and different tokens in the same request go to different experts, the serving engine cannot lazily load experts on demand without wrecking latency. So it doesn't. As HuggingFace's MoE writeup puts it, MoEs "require high VRAM as all experts are loaded in memory."

The consequence is blunt: "all parameters need to be loaded in RAM, so memory requirements are high." For Mixtral 8x7B, that means holding all 47B parameters resident even though only 13B are touched per token - your GPU tier is chosen as if the model were dense at its total parameter count. The MoE structure buys you speed, not a smaller memory bill. If you are sizing GPUs for any large model, our 2026 GPU Buyer's Guide walks through matching model footprint to hardware tier.

The upside: faster inference than a dense model of the same param count

The VRAM cost buys something genuine. "Although a MoE might have many parameters, only some of them are used during inference. This leads to much faster inference compared to a dense model with the same number of parameters." A dense 47B model would push all 47B through the matmuls for every token; Mixtral pushes 13B. That is the whole point of the architecture - you get the quality of a large model at the per-token compute of a much smaller one.

The trap is comparing the MoE only to its own total parameter count. Compared to a dense 47B, Mixtral is a bargain on compute. The question that actually decides your cost, though, is a different comparison - and it's the one nobody publishes.

Serverless billing: you rent the whole resident footprint plus cold start

On a serverless GPU platform, billing follows two things: the GPU tier that holds the resident footprint, and the time you spend paying for it, including cold starts. The footprint is set by total parameters, as above. The cold start is set by how long it takes to pull those parameters into GPU memory.

We don't have a published cold-start figure for a 671B MoE, so treat this qualitatively: cold start scales with the bytes you must move into VRAM, and an MoE moves all of them. For calibration on the small end, loading a Llama 3 8B TensorRT-LLM engine "takes roughly ~10-15s to load the model into GPU memory" - in our testing on Cerebrium. A model with an order of magnitude more resident weight will not load in that window. The practical implications for cold starts on serverless GPUs are covered in depth in our guide to serverless GPU cold starts, and the network component of the round trip in network latency in the AI inference pipeline.

The reason serverless still wins for bursty workloads despite cold starts is idle billing: a resident 47B or 671B footprint that scales to zero between traffic bursts costs nothing while idle, whereas a dedicated instance bills the whole time. We work through that trade-off in LLM inference cost at scale.

MoE vs a dense model of equal quality: the comparison nobody publishes

This is the framework, and the gap is the point. The honest question is not "MoE 47B vs dense 47B" - it's "this MoE vs the dense model that reaches the same quality." That dense equivalent is usually far smaller than the MoE's total parameter count, because MoE spends parameters to buy quality cheaply on compute, not on memory.

So the real cost decision is a three-way ledger no vendor benchmark lays out end to end:

• Compute per token - MoE wins (13B vs a dense model's full width).

• Resident VRAM / GPU tier - the dense-equivalent model often wins, because it may fit a smaller, cheaper GPU while the MoE's total footprint forces a larger one.

• Cold start and idle economics - favors whichever model has the smaller resident footprint, which again is usually the dense equivalent.

No published numbers put a specific dense-equivalent model head-to-head with Mixtral or DeepSeek-V3 on all three axes at once, so you have to measure it on your own traffic. That measurement gap is the value here: run your candidate MoE and a strong dense model on the same serverless GPU, measure time-to-first-token, throughput, GPU tier, and idle cost, and compare total cost per successful request - not parameter counts. Our own framework benchmark shows what that measurement looks like: on a single H100, batch size 1, 256 input tokens, total round trip to the platform, we measured a lowest TTFT of 123ms (vLLM) and peak throughput of 460 tokens per second (SGLang, batch 64) - see the full vLLM vs SGLang vs TensorRT-LLM benchmark.

Engineering the footprint down

The total-parameter footprint is not fixed destiny; architecture and serving choices move it. DeepSeek-V3 is the reference case: "to achieve efficient inference and cost-effective training, DeepSeek-V3 adopts Multi-head Latent Attention (MLA) and DeepSeekMoE architectures." MLA shrinks the KV-cache footprint at inference time, and DeepSeekMoE restructures the expert layout - together they make a 671B-total model tractable to serve. We wrote up a serverless, OpenAI-compatible endpoint for exactly this family in deploying DeepSeek-R1.

Beyond the model itself, quantization drops bytes-per-parameter, the right serving framework changes both TTFT and throughput on identical hardware, and speculative decoding raises tokens-per-second without adding resident weight - see speculative decoding on serverless GPUs. Fine-tuning a smaller model to your task can also erase the need for the big MoE entirely; DistilLabs did this on Cerebrium and "reduced their inference costs by 50%, and increased accuracy from 83% to 92%," as covered in how DistilLabs cut inference costs 50%. For task-specific serving patterns, see serving fine-tuned LLMs on serverless GPUs.

Decision checklist

• Size VRAM off total parameters, not active. 47B for Mixtral 8x7B, 671B for DeepSeek-V3. Active parameters predict speed, never footprint.

• Pick the GPU tier that holds the whole resident model. All experts stay in memory; there is no lazy-loading escape hatch.

• Budget cold start proportional to bytes loaded. ~10-15s covers a Llama 3 8B; a large MoE is much longer. Use scale-to-zero to offset it on bursty traffic.

• Compare against the dense model of equal quality, not equal parameter count. That is the comparison that actually decides cost.

• Measure cost per successful request on your own traffic - TTFT, throughput, GPU tier, idle - before committing to an MoE.

• Engineer the footprint down with MLA-style attention, quantization, the right framework, and speculative decoding before buying a bigger GPU.

MoE is a compute-saving architecture priced like a memory-heavy one. On a serverless GPU, the invoice follows the memory. If you're evaluating where to run it, our serverless GPU provider overview is a good next stop.