Analyzing Unstructured Data in BigQuery with ObjectRef

Historically, teams working with unstructured data have had to manage separate pipelines for structured and unstructured data, and for analytics versus AI/ML use cases. Running inference on files in Cloud Storage required orchestrating external machine learning workloads, then stitching structured outputs back into the warehouse for analysis — an approach Google describes as operationally heavy and prone to database misalignment. With ObjectRef and multimodal data analytics in BigQuery now in General Availability, that gap is bridged natively. ObjectRef is a built-in, predefined STRUCT data type pointing to objects outside BigQuery; storing these pointers in standard BigQuery tables alongside traditional columns makes unstructured files first-class data citizens that can be passed directly to Gemini models in a single SQL or natural language query, combined with relational columns, metadata, timestamps, and custom agent instructions.

The design rests on four advantages. The zero-copy architecture keeps high-volume media in Cloud Storage, respecting data gravity and avoiding duplication. ObjectRefs can be passed into BigQuery Python UDFs, letting teams bring open-source libraries such as OpenCV to the data instead of moving it. Because pointers track specific GCS generation numbers, inference stays reproducible even if the underlying file changes after a row is written. And the same BigQuery security policies and IAM controls govern structured columns and their referenced files.

A pattern Google calls full-fidelity ETL has emerged since the preview: building extraction, transformation, and loading pipelines entirely in SQL. Gemini within BigQuery extracts features and annotations into structured columns while the original ObjectRef remains anchored in the same row, which Google calls dual-layer analytics. Structured outputs power fast filtering, reporting, and BI dashboards, while conversational analytics agents can fall back to the original object for context-heavy reasoning. For example, sentiment scores and topic tags from customer call recordings can flag at-risk accounts on a BI dashboard, while an agent evaluating escalation processes can analyze the original audio for nuance.

Adoption evidence includes a quote from Snap data scientist Carlos Ortiz, who says Snap moved from simple random sampling to post-stratification sampling by leveraging ObjectRef with BigQuery AI, improving the efficiency of human review. Other production workloads cited are content moderation, proactive threat detection, inventory compliance, and customer experience analysis.

The GA release adds three capabilities. End User Credentials let analysts query unstructured data with their personal IAM identity for ad-hoc exploration, while BigQuery connections remain supported and are the recommended best practice for service-account-driven production pipelines. ObjectRef metadata can now come from BigQuery object tables, Cloud Storage insights datasets, or raw string URIs in standard tables. Native array support lets callers pass multiple ObjectRefs into a single Gemini call, enabling document comparison, image sequence analysis, and batching.

The article demonstrates three SQL workflows: call center audio analysis using OBJ.MAKE_REF() to detect audio format and AI.CLASSIFY() to categorize call types; product compliance filtering using AI.IF() inside WHERE clauses, with Optimized Mode as the default to reduce token consumption; and phishing detection that passes screenshots of an official login page and a suspicious page into one Gemini call. ObjectRef is generally available on all BigQuery editions and on-demand pricing tiers.

Analyze unstructured data in BigQuery with ObjectRef

View Original