Sandboxing Agent-Generated Code on Serverless GPU

Almost every published guide to sandboxing agent-generated code is, if you read it closely, a CPU guide. It reaches for a Firecracker-style microVM by default, treats cold-start latency as the only tradeoff worth discussing, and stops there. That advice is fine until the code your agent writes needs to touch a GPU - at which point the assumptions underneath it quietly stop holding.

This guide is about containing code written by agents that must run on an accelerator - not about serving inference to agents. If you are a Lead ML Engineer standing up a serverless GPU environment where an LLM proposes and then executes arbitrary Python, you need a framework that treats the compute boundary and the authority boundary as two different problems. They are.

Why the standard agent-sandbox advice is a CPU answer

The canonical answer to "how do I run untrusted, agent-generated code?" is a microVM. Firecracker is the reference implementation because it does exactly what the serverless world wants: it combines "the security and workload isolation properties of traditional VMs with the speed, agility and resource efficiency enabled by containers," and it can "safely run workloads from different customers on the same machine." For CPU-bound code that is close to a complete answer.

The reason it feels complete is that a CPU workload lives entirely inside the guest. The kernel is virtual, the devices are virtual, and the only real cost of adding isolation is the milliseconds it takes to boot. So the entire debate collapses into cold-start time - which is why every CPU sandbox guide reads like a cold-start guide.

What actually changes when the sandboxed code needs a GPU

The device is passed through, not virtualized

A GPU is not a virtual device the hypervisor can fabricate. To let sandboxed code run CUDA, the physical accelerator (or a partition of it) is passed through into the guest. That passthrough is the thing that makes the workload useful, and it is also the thing that punches a hole in the neat "everything is virtual inside the guest" model the CPU answer relied on.

The driver is shared with the host

Once the device is passed through, the guest is talking to a driver stack that shares state with the host. The isolation boundary that a microVM draws so cleanly around CPU and memory becomes porous at exactly the point the GPU enters. Your threat model has to account for a driver surface that both the tenant and the host touch - a consideration that simply does not exist for pure-CPU sandboxing.

Isolation has a measurable cold-start price

On CPU, cold-start is an annoyance. On GPU, it is the dominant cost, because you are not just booting a guest - you are initializing CUDA context, loading multi-gigabyte weights, and warming the driver. The isolation choice you make sits directly on that critical path. In Cerebrium's own snapshotting work, we measured a 71% average cold-start reduction across a six-workload benchmark suite, with reductions as high as 88% on vLLM - evidence of how much runtime state a GPU sandbox has to reconstruct before it can do anything. The isolation layer either helps you snapshot that state or fights you on it.

gVisor - intercepting the syscalls instead of handing over the kernel

The userspace-kernel model

gVisor takes a different tack from a microVM. Per its docs, "gVisor intercepts application system calls and acts as the guest kernel, without the need for translation through virtualized hardware. gVisor may be thought of as either a merged guest kernel and VMM, or as seccomp on steroids." There is no full virtual machine - there is a userspace kernel standing between the workload and the host.

Cold-start behavior

Because there is no guest kernel to boot and no fixed block of guest physical memory to stand up, gVisor gives you "a flexible resource footprint (i.e. one based on threads and memory mappings, not fixed guest physical resources) while also lowering the fixed costs of virtualization." For a GPU sandbox that spins up per request, a lower fixed cost of isolation is exactly what you want on the critical path.

The tradeoff

gVisor is explicit that the model "comes at the price of reduced application compatibility and higher per-system call overhead." For GPU workloads that is not a footnote: CUDA is syscall- and ioctl-heavy, so "higher per-system call overhead" and "reduced application compatibility" can bite precisely where your accelerator code lives. You buy a lighter isolation boundary and pay for it in compatibility risk.

Firecracker microVMs - VM-grade isolation at container speed, and the GPU wall

Why microVMs became the serverless default

Firecracker earned its default status on numbers. With "a minimal Linux kernel, single-core CPU, and 128 MiB of RAM, Firecracker supports a steady mutation rate of 5 microVMs per host core per second" - 180 microVMs per second on a 36-core host. That density plus true VM isolation is why serverless CPU platforms standardized on it.

Where it breaks for GPU

The microVM's strength is a hard boundary around virtualized hardware. A GPU is not virtualized hardware - it is passed through, and the driver is shared with the host. So the very property that makes Firecracker attractive on CPU is the property that a GPU passthrough compromises. You keep the boot-speed benefits, but the clean isolation story develops an asterisk at the device.

Cold-start vs gVisor

A microVM's fixed footprint (guest kernel, fixed guest memory) is the cost gVisor was designed to avoid. For a serverless GPU function that scales to zero and back, that fixed cost recurs on every cold start. gVisor's thread-and-mapping footprint can win here - at the compatibility cost above. This is the central compute-boundary tradeoff.

Choosing the compute boundary for serverless GPU

There is no single right answer; there is a decision. Choose a microVM when tenant separation is your hardest requirement and you can absorb the fixed cold-start footprint - and lean on snapshotting to claw it back. Choose gVisor when per-request startup dominates your economics and your CUDA workload stays inside its compatibility envelope. Either way, cold-start is engineering, not fate: we cut machine boot time by 83% by reworking node startup on AWS, detailed in our custom-container-image work, and Cerebrium's serverless GPU cold starts land in the 2-4 second range. The broader platform tradeoffs are covered in our guide to choosing a serverless GPU platform and in GPU inference for AI agent workloads.

The compute boundary is not the authority boundary

Here is the mistake even careful teams make: they pick an isolation runtime and call the problem solved. But a perfectly sandboxed process that still holds a valid database credential can do enormous damage without ever escaping its sandbox. The compute boundary limits where code runs. The authority boundary limits what it is allowed to do. You need both.

RBAC and capability boundaries

Start by constraining capabilities regardless of code content. As one practitioner framing puts it, "since everything can be capability constrained using RBAC," you can scope agent-generated endpoints so one role has read access to all databases while another has full CRUD and a third has only read-write on a single database. The sandbox never has to reason about the code - the capability grant already bounds the blast radius.

LOGOS-1 runtime authority

The deeper principle is separating cognition from execution. The LOGOS-1 proposal states it directly: "Capability != Authority, AdaptiveState != Authority, Proposal != Execution... the language model remains the cognitive component: it can reason, retrieve memory, learn procedures, evaluate evidence, and propose actions. But execution authority is deliberately kept outside adaptive model state." For GPU sandboxing this means the model that writes the CUDA code is never the component that authorizes it to run against real resources.

Authorization revalidation timing

The last gap is time. An agent workflow can pause between approval and execution, and "if the current source of truth for authorization can change after approval but before actual execution," you have a race: authority valid, action proposed, call approved, permission revoked, run resumes, execution begins against a grant that no longer exists. The fix is a fresh authorization check at execution time, or strict approval-to-execution atomicity. On serverless GPU this matters more, not less, because scale-to-zero naturally inserts a gap between approval and the cold-started execution.

Putting it together - a reference decision for containing GPU-bound agent code

Layer the two boundaries deliberately. First, pick a compute boundary: microVM for maximum tenant isolation, gVisor when per-request cold-start economics dominate and your CUDA path tolerates the syscall overhead - and treat GPU passthrough plus the shared driver as an explicit part of your threat model either way. Second, wrap that runtime in an authority boundary: RBAC-scoped capabilities so the code can only touch what its role permits, a LOGOS-1-style split so the proposing model never carries execution authority, and a fresh authorization revalidation at the moment of execution to close the approval-to-execution race. Third, make cold-start an engineering target rather than an accepted cost, using snapshotting to reconstruct CUDA state fast. Get all three right and you have contained agent-generated GPU code without paying for isolation you did not need or trusting a boundary that a passed-through device quietly breaks.

For related depth, see our work on serverless GPU cold starts and the Cerebrium run execution model, which runs your code in the cloud within ~2 seconds. If you are still evaluating where to run any of this, our overview of the serverless platform landscape and how to choose a serverless provider cover the surrounding decisions, and prefix caching for multi-turn LLM agents goes deeper on keeping agent state warm across turns.