
Vodafone Cuts Data Pipeline Costs 30% via Data Engineering Agent

Vodafone Czech Republic describes how it used Google Cloud’s Data Engineering Agent to modernize BigQuery stored procedures and build Dataform pipelines. The agent, available in BigQuery pipelines and Dataform and powered by Gemini, takes natural-language instructions together with code supplied in an instruction file. At Vodafone, many existing stored procedures are a legacy of an earlier on-premises data warehouse migration; some contain 2,000 to 3,000 lines of code and run sequentially. The modular Dataform approach replaces those monoliths with small .sqlx files, strict logic reuse through functions and ref(), built-in automated tests such as unique, not-null, and referential-integrity checks, auto-generated documentation with lineage, and automatic execution based on a DAG.
Using the Data Engineering Agent, Vodafone converted stored procedures into modular Dataform pipelines. The agent parallelized code into independent steps, improved readability and performance, and made it possible to execute only selected actions when debugging instead of running an entire procedure. It also created BigQuery materialized views for intermediate tables so data used by downstream processes is cached, which speeds up execution and lowers cost. In the new setup, Dataform handles end-to-end orchestration, pipeline dependencies, and execution logs, replacing the previous manual metadata tables, checkpoints, logs, and a spreadsheet that tracked dependencies between jobs.
The article reports measurable results: the agent achieved 1:1 logic parity with the original stored procedures while cutting migration effort by 90% compared with traditional manual methods. Post-migration benchmarks show the new pipelines run about twice as fast, and instructing the agent to cache intermediate tables reduced costs by 30%. Vodafone also highlights a trade-off between parallelization and caching: a sequential stored procedure can keep BigQuery‘s base-table cache for the first thread, but splitting logic into parallel streams can negate that benefit and cause full table scans for each thread. Creating materialized views for intermediate tables pre-cached the data and kept parallel queries performant and cost-efficient, and giving the agent specific instructions about which staging-table areas to parallelize made the parallelization more efficient.
Vodafone is planning to scale the agent across markets including Czech Republic, Romania, Ireland, Germany, and the United Kingdom. Planned investments include a standardized Dataform template repository, deeper integration into developer environments such as VS Code and Antigravity, federated CI/CD governance, and systematic migration of legacy workloads such as SAS code, BTEQ scripts, and MicroStrategy into modern Dataform pipelines.

