
Cross-cloud conversational analytics with borderless Lakehouse

Google Cloud’s borderless Lakehouse is presented as a way to run analytics across data spread across Databricks, Snowflake, AWS Glue, and BigQuery without copying data into a central warehouse. The post frames traditional scheduled pipelines as brittle: they cause synchronization lag, recurring storage costs, variable network egress fees, and break when schemas change. The alternative is to keep data in place and federate remote catalogs that implement the open Apache Iceberg REST catalog specification.
Under the hood, the Lakehouse runtime catalog separates metadata synchronization from storage access. It periodically syncs schemas, snapshot pointers, and table definitions from remote catalogs based on a refresh schedule, while at query time a compatible engine reads Parquet files directly from remote object storage using temporary credentials issued by the remote platform. All query processing runs in Google Cloud compute, so remote Databricks clusters, Snowflake virtual warehouses, and AWS compute instances are not invoked. Authentication follows a three-step pattern: create the federated catalog, inspect its configuration to retrieve the generated service account, and grant access permissions inside the remote platform.
The Databricks example is the most detailed. You prepare a Unity Catalog schema associated with an external location, enable External data access at the metastore level, and create a Databricks service principal. Then you create the federated catalog in Google Cloud with gcloud, describe it to get its assigned service account, grant EXTERNAL USE SCHEMA to the Databricks service principal, and configure the federation policy on that principal with Google’s issuer URL, the numeric subject ID, and the audience https://accounts.cloud.databricks.com.
Once catalogs are federated, four-part paths project.catalog.namespace.table address tables. The post shows a join across native BigQuery sales transactions, customer records in Snowflake, product lines in Databricks, and store locations in AWS Glue. BigQuery processes the join in place; Snowflake identifiers are uppercase while Databricks and AWS Glue paths remain lowercase.
Because federated Iceberg tables behave like native BigQuery tables, conversational analytics can be pointed at them. After saving the federated join as a logical view, users can ask natural language questions such as ‘Who are our top customers by total sales revenue, what products did they buy, and which stores did they shop at?’ BigQuery evaluates the schema across remote catalogs, generates the SQL, executes it, and returns a response.
The post also covers production caveats. The –refresh-interval flag controls metadata sync with a minimum of 300 seconds; if a sync cycle overruns, BigQuery skips the next scheduled run to avoid task queuing. BigQuery caches retrieved cross-cloud data blocks in the region where the federated catalog resides, which reduces network transfer on repeated queries but needs to be checked against data residency policies. Cache blocks use Google-managed encryption keys; if an organization enforces the constraints/gcp.restrictNonCmekServices policy, BigQuery bypasses the cache and reads from remote storage for every query. Cache usage can be inspected in the objectStorageStats section of BigQuery job statistics by comparing cacheBytesRead against objectStorageBytesRead.
Snowflake and AWS Glue setup appear in the appendix. Snowflake Horizon exposes managed Iceberg tables through an open REST catalog interface; a reader role and traversal permissions are needed, and a Snowflake service user is bound to Google’s numeric identity through Workload Identity Federation. AWS Glue configuration requires an IAM role with a 12-hour session duration, a permissions policy covering Glue catalog and S3 data access, and attention to AWS Lake Formation: the IAMAllowedPrincipals group must have access to target tables, or Lake Formation will block the role. To avoid premature background refresh failures while trust policies propagate, the catalog is initialized without a refresh schedule, and the IAM trust policy is updated with the verified Google Cloud subject ID afterward.
Next steps point to the borderless Lakehouse architecture documentation, a codelab for step-by-step federation, and working with a Google Cloud account team. The post does not claim support for every configuration; it explicitly defers to the documentation for supported regions and credential configurations.


