Google Cloud announced gcs-analytics-core, a new open-source Java library that centralizes and accelerates analytics optimizations for Google Cloud Storage (GCS). The library sits between analytics engines (Apache Spark, Trino, Apache Hive) and the GCS Java SDK, intercepting read calls to inject performance enhancements without requiring framework-specific tuning. It is available natively in the Apache Iceberg Java runtime from version 1.11.0.

Key technical optimizations include Vectored I/O (threaded), which fetches multiple data ranges in parallel within a single operation to reduce the overhead of GCS calls, and Smart Parquet prefetching, which automatically reads the Parquet file footer (typically 50–100 KB) in a single chunk to avoid multiple network calls when engines seek backward for metadata. For Apache Iceberg users, the library integrates into the GCSFileIO implementation, replacing traditional sequential reads with parallelized strategies.

The first major integration is in Apache Iceberg 1.11.0+. With Iceberg configured to use GCSFileIO, the library automatically provides Parquet footer prefetching and multi-threaded vectored reads — no custom tuning required. The library is compatible with all Iceberg catalogs, including REST catalog, Hive, and other metadata management systems, decoupling performance optimizations from the catalog layer.

To validate improvements, end-to-end TPC-DS benchmarks were run on an Apache Spark cluster with Iceberg and GCSFileIO, comparing the new library against the default GCSFileIO implementation using sequential vectored reads. Results across dataset sizes from 1 GB to 10 TB showed consistent scan time reductions: 71.51% at 1 GB, 48.48% at 10 GB, 40.98% at 100 GB, 35.86% at 1 TB, and 18.40% at 10 TB. Execution time improvements ranged from 32.61% at 1 GB down to 1.58% at 10 TB, reflecting that the library alleviates I/O bottlenecks, letting compute engines spend more time on processing.

To get started, users need Apache Iceberg Spark runtime 1.11.0+ and the iceberg-gcp-bundle 1.11.0+, configure the catalog to use GCSFileIO, enable the gcs-analytics-core optimization flag (spark.sql.catalog.$CATALOG_NAME.gcs.analytics-core.enabled=true), and enable vectorized I/O (spark.sql.iceberg.vectorization.enabled=true). The library is open source on GitHub at GoogleCloudPlatform/gcs-analytics-core, with a design document for architectural details. The team encourages users to test on their own datasets and share results.