
Unify public and private data with Data Commons on Spanner Graph

Today, Google announced the general availability of Data Commons on Spanner Graph and the preview of a new Data Commons Platform that unifies private enterprise knowledge with public statistical datasets. The overarching Data Commons project aggregates over 400 billion data points from 100+ authoritative providers (UN, World Bank, US Census, Eurostat, WHO, NOAA) using Schema.org definitions, covering domains such as agriculture, demographics, economy, environment, and health.
The architecture has migrated from a Bigtable-based caching layer to a native graph model on Spanner Graph, which provides SQL-like interfaces, high availability, horizontal scale-out, multi-region transactional consistency, and native GQL support. By adopting a multi-entity schema, entities become nodes and domain links become dynamic graph edges, eliminating pre-computed caches and enabling complex relationship queries directly in the database. This shift also allows incremental updates to specific datasets without full database refreshes, using stale reads for consistent snapshots during ingestion.
Key benefits include: unified storage with incremental data imports, dynamic graph traversals via GraphRAG for multi-hop queries (e.g., continent → country → state → county → city), consistent data snapshots via Spanner TrueTime, and operational analytics at scale using Spanner’s columnar engine and BigQuery federation with Data Boost for isolated complex aggregations.
The platform now supports SDMX 3.0, an ISO standard for statistical data exchange, enabling out-of-the-box integration with tools like Tableau, Flourish, and Observable via SDMX-JSON and SDMX-CSV 2.0 formats. Two high-value endpoints are provided: an availability API for programmatic discovery of dimensions and variables, and a data API that retrieves observations with named parameters to avoid breaking on dimension additions.
For organizations wanting private instances, the new architecture resolves legacy scaling limits. Developers can deploy a private instance of the Data Commons Platform using the same technology as Google’s public instance, retaining full data control and limiting access. By federating across the public knowledge graph and a private graph, users can combine public data (GDP trends, demographics, employment) with enterprise data (sales, store performance, supply chain) using natural language queries, without data duplication or loss of isolation. The source explicitly states this enables use cases such as retail enterprises optimizing merchandise distribution by contrasting macroeconomic indicators with company transactions.


