Google Expands TPU Portfolio with TPU 8tx and TPU 8i Max

Google has expanded its eighth-generation TPU portfolio with two new systems: TPU 8tx for confidential inference and TPU 8i Max for massive context windows. The announcement brings the generation to four systems — TPU 8t, TPU 8i, TPU 8tx, and TPU 8i Max — the first time Google has introduced four TPU systems in a single year. TPU 8t and TPU 8i, introduced at Google Cloud Next in April, deliver up to 2.7x better training performance and 80% better inference price performance, respectively, versus the previous generation. Google says TPUs are now a rapidly growing business with nearly 100 customers, including frontier labs, trading firms, retailers, and universities; use cases range from training and serving Gemini models to agentic workloads, Google Cloud services, and customer-owned data-center deployments.

Google distinguishes a new chip from a new system: a system is how compute, networking, and storage are packaged around the chip. The company’s foundational bet on system balance and modularity, architected as building blocks within the AI Hypercomputer, allows engineers to adjust packaging, high-speed memory, optical networking, and security, delivering specialized supercomputers in months rather than years. The two new systems address two urgent challenges: hardware-level data security and million-token context scale.

TPU 8tx is built on the same silicon as TPU 8t and paired with custom AMD CPU hosts. Security is built into the hardware layer, keeping sensitive data and model weights protected and isolated while in active use, rather than only when stored or in transit. By adopting the low-latency Boardfly network created for real-time reasoning, TPU 8tx delivers up to 2x lower latency compared to TPU 8t with a near-zero performance penalty. It also supports small-scale training, private fine-tuning, distillation, and agent reinforcement learning on a single system, without moving sensitive data between clusters.

TPU 8i Max addresses the memory wall by taking two low-latency TPU 8i chips and combining them onto a single package. It scales compute and memory in lockstep, delivering double the compute, double the SRAM, and double the HBM bandwidth of TPU 8i, so processors remain equipped with essential data and large workloads do not stall. Because it fits twice the silicon within the existing rack footprint, it offers the highest compute density per rack and targets massive context windows.

On the software side, Google open-sourced TorchTPU, so PyTorch code and Hugging Face models run natively on TPUs, with out-of-the-box backends for vLLM and SGLang. The TPU agent suite (in preview) plugs into Claude Code, Cursor, and Google Antigravity to automate onboarding, diagnose bottlenecks, and autotune kernels; the open-source XLA Serving Engine allows Python-trained models to be served in a low-latency C++ runtime across TPUs, GPUs, and CPUs. Google reports state-of-the-art price-performance with the native vLLM-TorchTPU stack on InferenceX and AgentX benchmarks, while maintaining high model accuracy, and says upcoming software optimizations will drive results further. Agent Compute Engine suspends idle agent sessions and shifts working memory off chip in milliseconds, and TPU Sync, an open-sourced model-state layer, turns chip HBM, host DRAM, shared storage, and the network into a single memory hierarchy, tiering KV caches off chip, streaming context across disaggregated prefill and decode nodes, and synchronizing RL weights to cut KV transfer latency and time-to-first-token by more than 90%.

Cloud Pathways, the Google DeepMind runtime behind Gemini, is now generally available on GKE, orchestrating large-scale TPU systems across multiple slices as a single local machine, with colocated Python sidecars, sub-minute slice hot-swapping, and elastic resharding. GKE dynamic slicing, also generally available, decouples physical hardware provisioning from logical slice formation using optical circuit switching, enabling on-demand super-slicing and sub-slicing; this slashes job startup times by up to 5x and accelerates fault recovery by up to 4.5x. Google attributes the ability to ship four TPU systems and a unified, open software stack in one year to co-design with Google DeepMind, including AlphaChip reinforcement learning and Gemini-assisted chip layout.

Introducing TPU 8tx and TPU 8i Max

View Original