Great comments! Another way to look at Dataflow pricing as it related to this type of technology. Let's just assume for a second Dataflow and other similar tec

Great comments! Another way to look at Dataflow pricing as it related to this type of technology. Let's just assume for a second Dataflow and other similar technologies execute your pipeline on your data within the same resource-time amount. Dataflow lets you set an upper bound on these resources, and lets you auto-scale, with essentially per-minute granularity. Alternatively, you can create a cluster of fixed resource amount. Cost of Dataflow is $0.01 per hour per CPU, in addition to those resources. Dataflow's model hence either guarantees 100% resource efficiency, or you specifically opt into the upper bound of spend. A deployment model that lets you "spin up a cluster and pay for it" is intrinsically less efficient. I wrote a blog on this WRT BigQuery, but Dataflow is right there [0]. [0] https://cloud.google.com/blog/big-data/2016/02/understanding-bigquerys-rapid-scaling-and-simple-pricing https://cloud.google.com/blog/big-data/2016/02/understanding... (Work on Google Cloud)