EU Data Residency for LLM Inference on Serverless GPU

Choosing an EU region for your database answers where data rests. It says almost nothing about where a prompt is processed, where the KV cache lives while a model generates, what a provider retains afterward, or where traffic goes when your primary region saturates. Residency for serverless GPU inference is a distinct surface from storage residency, and GDPR's own language is the reason: it regulates processing, not just the disk the data ends up on.

This is a decision guide for engineering leaders, not a legal brief. It names the four inference-specific residency surfaces, grounds them in the parts of GDPR and EDPB guidance that actually apply, and gives you a framework and a set of provider questions to pin each surface before you commit.

Why "we picked the EU region" doesn't answer the residency question

Data has three states: at rest, in transit, and in use. Selecting an EU region governs the first. Inference lives almost entirely in the other two - a prompt in flight to a GPU, tokens held in memory during generation, activations resident in VRAM.

GDPR does not scope its obligations to storage. Article 44 applies to personal data "which are undergoing processing or are intended for processing after transfer," meaning the moment of processing is itself in scope, not merely the moment of storage. And under the EDPB's definition, a transfer occurs whenever a processor subject to GDPR "makes personal data available to another organisation" located outside the EEA - no persistence required. A prompt streamed to a GPU in a non-EEA region is a transfer even if nothing is ever written to disk. "We picked the EU region" is a storage answer to a processing question.

The four inference-specific residency surfaces

1. The prompt in flight and the compute location. The request body - often the most sensitive payload you own - travels to wherever the GPU actually executes. If that node sits outside your declared region, you have a transfer at inference time regardless of where the response is stored.

2. The KV cache as data-in-use. During generation the model holds attention state in GPU memory. That cache is personal data in use, resident on a specific physical node in a specific region. Prefix and prompt caching, which we cover in prefix caching for multi-turn LLM agents, can also persist derived state across turns - extending the window in which that data is live.

3. Provider retention and logs. Request logs, prompt/response captures for debugging, and abuse-monitoring buffers are all processing you may not have authorized. Under Article 28 a processor may act "only on documented instructions from the controller" - silent retention outside those instructions is a gap.

4. Cross-region routing under load - the highest-risk surface. This is the one storage policies never touch. When a region saturates, autoscalers and failover logic can route the next request to spare capacity elsewhere. A correctly configured EU deployment can silently spill a prompt to a non-EEA node at 3am under a traffic spike. It is invisible in steady-state testing and it is exactly the scenario Article 44 governs.

The legal frame a VP needs

You do not need to be a lawyer here, but three anchors let you brief one accurately.

Article 28 - the processor relationship. A controller must "use only processors providing sufficient guarantees to implement appropriate technical and organisational measures." Your serverless GPU provider is a processor. The residency guarantees have to live in the DPA, and the processor must act only on documented instructions - including "with regard to transfers of personal data to a third country."

Article 44 - transfers are about control, not just location. Chapter V restrictions exist, in the EDPB's words, so that "the level of protection of individuals granted by the GDPR remains the same" wherever data goes. The EDPB's three cumulative criteria for a transfer - GDPR applies, data is disclosed to another organisation, that organisation is outside the EEA - are what you test each surface against.

SCCs and their limits. Where there's no adequacy decision, transfers can proceed on "appropriate safeguards, including enforceable rights and legal remedies for individuals," and the modernised Standard Contractual Clauses under Regulation 2021/914 are the common instrument. But SCCs legitimize a transfer; they don't prevent one. If your architecture routes EU prompts to a US node, SCCs may make it lawful - they don't make it not-a-transfer, and they don't remove the diligence obligation. Note too that the EU AI Act came into force on 2 February 2025, adding a parallel obligation layer for in-scope systems.

How serverless GPU regions actually behave

Regions are stronger isolation boundaries than most teams assume. AWS documents that "each AWS Region consists of multiple independent and physically separate Availability Zones," that regions are "isolated and independent from other Regions," and - critically - that "the resources and data that you create in one Region do not exist in any other Region unless you explicitly use a replication or copy feature." So absent explicit replication, at-rest residency within a region is a strong default.

The catch is that serverless inference introduces movement the region boundary alone doesn't describe:

• Cold starts and pooling. Scale-to-zero platforms spin capacity up on demand. Where that capacity is drawn from, and whether a warm pool spans regions, determines where your first request actually runs. Cold-start engineering matters for latency - we measured memory snapshots reducing cold start time by more than 80% for some workloads - but the residency question is which region the restored container lands in. The mechanics are in reducing GPU cold starts with memory snapshots.

• Spillover. Under load, does the platform overflow to another region, or does it queue and cold-start more capacity in-region? That single behavior is the difference between compliant and not.

• Deployment-model spectrum. Fully managed multi-tenant, single-tenant in a pinned region, dedicated capacity, and self-hosted each trade convenience for control. See our note on multi-region deployments and the global-scale platform guide for how region pinning interacts with scaling.

A decision framework

Step 1 - Classify the data first. Not every prompt carries personal data. Segment workloads: no-personal-data (residency is a preference), pseudonymous, and identifiable/special-category (residency is a hard constraint). Over-constraining the first tier just burns money you could spend elsewhere, as we discuss in LLM inference cost at scale.

Step 2 - Match tier to architecture. Low tier can use the most convenient managed deployment. High tier needs a pinned single-region deployment with routing and spillover locked down, or dedicated capacity.

Step 3 - Pin every surface. Walk the four surfaces as a checklist: compute location pinned; KV cache confined in-region; retention and logging set to documented instructions; failover and routing constrained to EEA nodes only. A deployment that passes three and leaks the fourth is non-compliant.

Provider questions that expose residency gaps

• Compute and cache: In which region does the GPU physically execute? Does the KV cache ever leave that region? Is a warm pool region-scoped?

• Retention and deletion: What is logged by default, for how long, and where? Can retention be contractually disabled to match documented instructions under Article 28?

• Failover and routing: Under saturation, do you spill to another region or cold-start in-region? Can cross-EEA-boundary routing be hard-disabled? Will you attest to it in the DPA?

If a provider can't answer the routing question in writing, treat the highest-risk surface as unmanaged.

What "good" looks like - reference patterns per tier

• Low tier (no personal data): managed serverless, region chosen for latency and cost. Choose freely across serverless GPU providers; optimize for network latency in the inference pipeline.

• Mid tier (pseudonymous): EU region pinned, spillover disabled, retention minimized, SCCs in place with any non-EEA subprocessor. Consider AWS alternatives for AI workloads where region control is clearer.

• High tier (identifiable/special-category): single-region or dedicated capacity, no cross-region routing, zero request retention, DPA attesting compute and cache location. This is also where you scrutinize whether fine-tuned LLMs on serverless GPU keep weights and cache inside the boundary.

Residency for inference is won at the surfaces storage policy ignores. Classify the data, match the architecture, and pin all four - the prompt, the cache, the retention, and the routing.