
Optimizing GKE networking for large agentic AI workloads

The post explains how GKE Dataplane V2 handles a bottleneck that emerges when agentic AI workloads grow to extreme concurrency: network identity management. Autonomous agent workflows differ from static requests because they spin up and tear down large numbers of isolated code-execution sandboxes, often up to one million concurrent sandboxes in advanced research settings. At that scale, Kubernetes networking efficiency directly affects cost, since expensive GPUs and TPUs should spend time running models rather than waiting for sandbox initialization. The post focuses on how GKE Dataplane V2, powered by eBPF technology, keeps provisioning fast while preserving network security policies.
The core problem is that workloads commonly assign unique, dynamic labels to each pod, such as agent-id, timestamp, or claim-uid. GKE Dataplane V2 maps pod labels to numeric identities to enforce security policies, and agentic frameworks naturally generate highly unique labels for every sandbox. As a result, the system must provision a distinct identity for each pod, and the serial allocation process can briefly pause pod creation while eBPF maps are updated. At sufficient scale, this slows initialization and can approach the cluster-wide limit of 65,536 network identities.
To solve this, GKE Dataplane V2 separates identifying data from functional metadata. Fields that do not participate in service selector matching or core resource grouping should be moved from a pod’s labels to its annotations. Since GKE Dataplane V2 computes network identities from labels rather than annotations, moving high-cardinality metadata out of labels avoids per-pod identity generation entirely, resolving identity scaling limits while retaining visibility. The post also describes advanced Cilium label filtering: by applying exclusion filters, for example through the cilium-config-emergency-override ConfigMap in kube-system, operators can instruct GKE Dataplane V2 to ignore dynamic labels such as sandbox-name-hash or timestamp. Even if a developer framework adds high-cardinality labels inadvertently, the dataplane can drop them automatically.
With these changes, identical agent types share a single Cilium pod identity, preventing linear identity growth and avoiding Cilium‘s 65,536 identity limit. The reported result is leading speed for fully isolated agents: 300 sandboxes per second with sub-second latency. A customer is quoted as using Agent Substrate on GKE to orchestrate hundreds of thousands of secure environments across clusters and to handle spikes of over 30,000 sandboxes on a single cluster.
Looking ahead, Agent Substrate on GKE natively integrates these Dataplane V2 recommendations while also bypassing Kubernetes scheduling limits. In benchmark validation running 200,000 agents, Agent Substrate sustained 500 activations per second at less than 500 ms suspend and resume latency. It can be deployed on existing GKE clusters to reduce idle compute cost for demanding AI workflows. The overall takeaway is that optimizing network identity calculation and filtering high-cardinality metadata reduces sandbox provisioning latency to sub-second levels, limits idle GPU time, and helps keep compute density high while controlling operational costs.


