
BigQuery and Looker bring observability to enterprise AI agents

Google is expanding AI agent observability and evaluation across Google Data Cloud, giving teams unified visibility into first-party (1P) data agents in BigQuery and Looker and custom third-party (3P) agents built with Agent Development Kit (ADK) or LangGraph. The post targets three operational challenges for agents in production: tracing multi-step reasoning while protecting sensitive data, controlling token consumption and query spend, and evaluating answer accuracy before prompt or schema changes reach users.
The core mechanism is a unified telemetry framework across Google’s Agentic Data Cloud: capture telemetry from the agent, land it in BigQuery tables the customer owns, and analyze it wherever teams already work. First-party Conversational Analytics agents emit telemetry automatically with no setup. Queries routed to BigQuery data agents in Gemini Enterprise through agent-to-agent (A2A) workflows, now GA, send Cloud Monitoring metrics, Cloud Trace spans, and BigQuery conversation logs into the agent’s project. Custom agents built with ADK or LangGraph stream the same structured telemetry into BigQuery through single-line SDK instrumentation.
Conversation logging in BigQuery is GA: managed first-party logging writes to customer-owned tables, so prompts and responses from Conversational Analytics traces can be inspected directly in the data warehouse. The Conversational Analytics API is also GA, with always-on admin audit logs, optional data-read audit logs in Cloud Audit Logs, VPC Service Controls, customer-managed encryption keys (CMEK), and regional or multi-regional data residency. For analysis, prebuilt agent monitoring dashboards in BigQuery Studio (GA) track usage, active sessions, top knowledge sources, and latency. System Activity dashboards for Conversational Analytics are GA in Looker (Google Cloud core) and Looker (original) for engagement and token usage, while user ratings and query review are in preview. Teams can also build reports from an open-source Data Studio template or deploy the Looker Block for Agent Analytics.
Enterprise observability means balancing diagnostic telemetry with privacy. Google Data Cloud separates operational telemetry from conversation content so admins can govern each layer independently. Cloud Monitoring aggregates metrics such as request volume, latency, and token usage; Cloud Trace records step-by-step spans, with observability scopes to consolidate telemetry across multiple agent projects. These operational metrics and sanitized first-party trace spans exclude customer core content: user prompts, model responses, internal thoughts, and generated SQL. Deeper transcript auditing is opt-in per agent. Governed conversation logging streams user prompts, responses, and turn-level telemetry into a customer-owned agent_events table via the BigQuery Storage Write API, inheriting dataset IAM permissions, CMEK, and VPC Service Controls. A fail-closed check before each turn skips logging whenever a knowledge source uses row-level security, column-level policy tags, or dynamic data masking; fail-open execution keeps chat sessions running if a logging error occurs.
First-party conversation logging intentionally strips low-level tool arguments and internal SQL, but custom ADK or LangGraph agents can instrument code with the open-source BigQueryAgentAnalyticsPlugin. Single-line setup streams full tool-invocation execution trees, time-series telemetry, and multimodal payloads into BigQuery. Because logs land natively in agent_events, no separate export pipelines are needed for root-cause analysis: teams can run BigQuery AI functions such as AI.GENERATE to summarize error patterns and AI.EMBED with VECTOR_SEARCH to cluster recurring issues, or use the open-source bigquery-agent-analytics Python SDK and Colab starter notebook for semantic drift clustering and LLM-as-a-judge sentiment analysis.
For cost control, each turn in BigQuery conversation logs records input, output, thought, cached, and total token counts alongside total turn latency, and Looker System Activity reports estimated input and output tokens. Turnkey dashboards track projected token usage and engagement. To prevent unexpected charges, per-query byte scan caps (big_query_max_billed_bytes) use dry-run estimates to block oversized queries without on-demand scan charges; daily input and output token quotas per project, user, or agent are in preview in the Cloud console and gcloud CLI.
On the evaluation side, the open-source tool Prism tests Conversational Analytics agents grounded in BigQuery tables or Looker Explores via the Conversational Analytics API, using reusable test suites, regex assertions, and multi-region agent discovery behind an access layer such as Cloud IAP. Prism can stream completed evaluation runs into day-partitioned BigQuery tables, placing pre-deployment benchmark scores alongside production agent_events logs for side-by-side analysis in BigQuery and Looker. Native first-party agent evaluations in BigQuery, built on the Gemini Enterprise Agent Platform evaluation service with bring-your-own-service-account support, are in preview via API and a guided Colab notebook.
The post closes with a three-step path: enable first-party observability by turning on Cloud Monitoring and Cloud Trace and opting in to per-agent conversation logging; instrument custom agents with the BigQuery Agent Analytics SDK and deploy Data Studio or Streamlit dashboards; then benchmark with Prism and configure byte scan caps and daily token quotas.

