gcs-analytics-core: Optimizing Iceberg and Spark on GCS

Data engineers often struggle with compatibility and performance across multiple analytics engines on cloud storage. Google Cloud has released gcs-analytics-core, an open-source Java library that centralizes read optimizations for Google Cloud Storage (GCS), targeting engines like Apache Spark and Trino via the Apache Iceberg runtime. The library sits between analytics frameworks and the GCS Java SDK, injecting parallelized I/O and prefetching strategies without requiring engine-specific tuning.

The core optimizations include Vectored I/O (threaded)—which fetches multiple data ranges in a single operation to reduce call overhead—and Smart Parquet prefetching, which reads footer metadata (50KB–100KB) in one chunk instead of multiple backward seeks. The first major integration ships with Apache Iceberg 1.11.0+ through the GCSFileIO implementation. TPC-DS benchmarks across 1GB to 10TB datasets show scan time improvements from 71.51% (1GB) to 18.40% (10TB) and execution time gains from 32.61% to 1.58%, with larger relative gains at smaller scales.

For builders, this library eliminates the need for framework-specific hacks to get good I/O performance on GCS. Enabling it requires only a few Spark configurations (e.g., spark.sql.catalog.$CATALOG_NAME.gcs.analytics-core.enabled=true) and the correct Iceberg runtime version. This is a practical, drop-in optimization for any data lake using Iceberg on GCS—especially useful for teams running diverse analytics workloads who want consistent storage-layer performance without reinventing the wheel.