Dataflow GA: Pause/Resume for Batch Jobs and RTX PRO 6000 GPUs

Google Cloud announced two Dataflow enhancements aimed at large-scale AI and agentic workloads. First, Pause/Resume for Dataflow batch jobs is now generally available. The feature targets long-running batch jobs that can run for days: previously, a failed job required a full retry, and users could not access data already processed before the failure, wasting compute and engineering time. Pause/Resume lets customers resume failed jobs instead of restarting them, and also allows pausing and resuming lower-priority batch jobs to dynamically reallocate accelerated compute such as GPUs and TPUs to higher-priority work like feature engineering and AI inference.

Second, Dataflow adds support for G4 VMs backed by NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs. According to Google, the RTX PRO 6000 Blackwell delivers significant performance gains over the NVIDIA L4 GPU, with 96GB vGPU memory and 1.6 TB/s of bandwidth. That makes it possible to run AI inference inside Dataflow with up to 70B+ parameter models while using existing Dataflow ML capabilities such as RunInference, right fitting, and GPU-enabled autoscaling, avoiding manual infrastructure tuning. The two features together are positioned to help optimize batch job costs and enable demanding AI workloads. No additional performance benchmarks or migration details are provided in the announcement.

New Dataflow features to enable large scale AI workloads

View Original