6 ms·

This blog post from 2 years ago from AWS has Redshift beating BigQuery pretty handily on TPC-H and TPS-DS: https://aws.amazon.com/blogs/big-data/fact-or-fiction

This blog post from 2 years ago from AWS has Redshift beating BigQuery pretty handily on TPC-H and TPS-DS: https://aws.amazon.com/blogs/big-data/fact-or-fiction-google-big-query-outperforms-amazon-redshift-as-an-enterprise-data-warehouse/ https://aws.amazon.com/blogs/big-data/fact-or-fiction-google... Has anything changed on BigQuery since then that would warrant rerunning those benchmarks? If Amazon’s largest cluster outperforms BigQuery then is your decision about which service to use just a cost calculation of dedicated per month for Redshift vs. projected units scanned per month on BigQuery?

fhoffa 8y ago
Let's say you are Twitter, Reddit, or the CTO of your current company. How do you choose your next analytical platform: a) Do you look at the benchmarks that each company produces, and choose the one that publishes the numbers that make them look the best? b) Do you ask "I have n data analysts with m different questions and I want to give them the most productive platform I can". Well, this is what Twitter chose: - How Twitter Migrated its On-Prem Analytics to Google Cloud https://www.youtube.com/watch?v=N3JAwCYGHU8 https://www.youtube.com/watch?v=N3JAwCYGHU8 Or listen to Nick Caldwell, Reddit VP Engineering, moving away from AWS to BigQuery: - "2017, which effectively brought us to the present system, we began forking all of our event data into BigQuery, after considering a lot of different alternatives" https://youtu.be/tKISLQ87GO8?t=426 https://youtu.be/tKISLQ87GO8?t=426 I prefer method b) :) Disclosure: I'm https://twitter.com/felipehoffa https://twitter.com/felipehoffa and I work for GCP
blaisio 8y ago
I've worked with both, and BigQuery is so much easier to work with. Redshift does give you more control (to a limited extent) but it takes a lot more work and specialized knowledge to perform simple tasks well. In the article you linked, they gloss over the part that takes by far the longest when working with redshift: setting up "compression, distribution keys on large tables, and sort keys on commonly filtered columns". With BigQuery, you don't really worry about that crap, or about reloading your terabytes of data if you made a mistake somewhere in the schema. You also don't worry about vacuuming, or running out of space, or taking down the database from excessive CPU usage. Do you want to have a team of people at your company whose job is just to keep Redshift running smoothly? Or do you want another team of analysts or engineers? I could see some companies being forced to use Redshift, especially if they're using S3 a lot, and I could see people saving money in some specialized use cases, but for most people BigQuery is almost certainly going to be cheaper and faster in almost every way.
- jimmytucson 8y ago
  > I could see people saving money in some specialized use cases, but for most people BigQuery is almost certainly going to be cheaper and faster in almost every way That’s exactly what the TPC benchmarks are designed to show: how different appliances perform under the same diverse set of generic workloads. As of 2 years ago, they show that Redshift is faster. > With BigQuery, you don't really worry about that crap The benefit this depends on your organization’s level of expertise. If you grok sorting and distribution then you can leverage those to increase performance—but it’s not a prerequisite. For example, in Redshift, when you bulk load data into a table, if the user didn’t specify a compression scheme in the table definition, Redshift will analyze the data and find a scheme that works best, and automatically apply it to the table for you. BigQuery almost certainly does something similar. The difference is, with BigQuery, you’re not invited to take part in that discussion. And you’re charged as if the data is uncompressed. Psychologically, this is a huge relief if you don’t (want to) know how compression works but rest assured you’re paying for it somehow. To draw a tired analogy, vehicles with automatic transmission still have to shift gears. If you’re driving to the grocery store, not having to worry about that is a win. But if you’re racing stock cars, you’re definitely going to want a stick.
  - deleted 8y ago
    [deleted]
  - vgt 8y ago
    Compute resources tend to be significantly more expensive than storage. Our approach is two-fold: - We finance compute resources required for you to ingest data into BigQuery. With Redshift, you pay for ingest directly via compute cluster consumption (again, more expensive than storage). This also increases your complexity due to on-cluster contention of resources between ingest and query. - Separation of storage and compute gives you lots of options. With BigQuery ,you don't need to attach relatively expensive compute just for the luxury of getting more storage. Spectrum helps somewhat, but ultimately with Redshift you don't even get to pay for storage - you pay for compute/storage combos. - BigQuery's Long-Term Storage is not an archival storage tier - it's a discount on storage, with identical performance and durability characteristics. At only $0.01 per GB per month. This is likely a result of origins of the two technologies. BigQuery is Dremel, written and operated by Google since 2006. Redshift purchased source code to an on-premise fork of Postgres.
havermeyer 8y ago
From today's release notes (https://cloud.google.com/bigquery/docs/release-notes#july_25_2018 https://cloud.google.com/bigquery/docs/release-notes#july_25...): > BigQuery has achieved approximately 5x improvement in TPC-DS 10T power run geomean over the past 18 months. Outside of that, BigQuery now provides better partitioned table functionality (https://cloud.google.com/bigquery/docs/partitioned-tables https://cloud.google.com/bigquery/docs/partitioned-tables) as well as clustering (https://cloud.google.com/bigquery/docs/clustered-tables https://cloud.google.com/bigquery/docs/clustered-tables), both of which can help to reduce costs and improve performance. In terms of pricing, many large customers have reserved slots (https://cloud.google.com/bigquery/docs/slots https://cloud.google.com/bigquery/docs/slots), where you pay a fixed monthly cost in exchange for dedicated resources, similar to running a large Redshift instance. As a disclosure, I work on the BigQuery team.
vgt 8y ago
Great question. At the time we had a hard time replicating results of that blog post. Here's some food for thought. - We recently announced that we've improved our TPC-DS 10T geomean by 5x over past 18 months [0]. The nice thing is that users never had to set maintenance windows or "upgrade their clusters" to get these performance improvements. - Here's a session by our eng lead and CTO of Looker discussing various performance improvements, including querying a PB of data in ~5 seconds and paying a fraction of a penny for it.[1] - Benchmarks are important, but they miss three key scenarios. ------ Effort - how much complexity is there to achieve maximum performance. As someone else stated in this thread, we feel that we differentiate here. ------ Maintenance - when data changes, new data comes in, you run DML, what happens to this pristine benchmark of yours? What is required to maintain this performance? Again, we feel that our offering is compelling (we don't ever have messy storage state that requires a vacuum, our compute is entirely stateless so re-sorting/re-distributing data is not needed, and BigQuery ingest never affects query capacity, even at PB/day ingest scale). ------ Real-world usage - a serially-executed set of queries poorly represents what happens in reality - high volatility, high concurrency workloads. Again, BigQuery has stateless compute, separation of compute and storage, and separation of compute and state. - customers - in addition to what's been stated already, PTAL at Kabam [2] & NYT/Yahoo/BlueApron [3] for their motivations for migrating from Redshift to BigQuery That said, lots of folks really like running on Redshift, and they've extended their platform with Spectrum and other bits. I'd invite you to find out what works best for you. Competition is good for the end user! [0] https://cloud.google.com/bigquery/docs/release-notes#july_25_2018 https://cloud.google.com/bigquery/docs/release-notes#july_25... [1] https://www.youtube.com/watch?v=6Nv18xmJirs https://www.youtube.com/watch?v=6Nv18xmJirs [2] https://www.youtube.com/watch?v=6Nv18xmJirs https://www.youtube.com/watch?v=6Nv18xmJirs [3] https://www.youtube.com/watch?v=TLpfGaYWshw https://www.youtube.com/watch?v=TLpfGaYWshw
- jimmytucson 8y ago
  Plenty to chew on here. Thank you
  - vgt 8y ago
    Please don't hesitate to reach out for follow ups

fhoffa 8y ago

Let's say you are Twitter, Reddit, or the CTO of your current company. How do you choose your next analytical platform: a) Do you look at the benchmarks that each company produces, and choose the one that publishes the numbers that make them look the best? b) Do you ask "I have n data analysts with m different questions and I want to give them the most productive platform I can". Well, this is what Twitter chose: - How Twitter Migrated its On-Prem Analytics to Google Cloud https://www.youtube.com/watch?v=N3JAwCYGHU8 https://www.youtube.com/watch?v=N3JAwCYGHU8 Or listen to Nick Caldwell, Reddit VP Engineering, moving away from AWS to BigQuery: - "2017, which effectively brought us to the present system, we began forking all of our event data into BigQuery, after considering a lot of different alternatives" https://youtu.be/tKISLQ87GO8?t=426 https://youtu.be/tKISLQ87GO8?t=426 I prefer method b) :) Disclosure: I'm https://twitter.com/felipehoffa https://twitter.com/felipehoffa and I work for GCP

blaisio 8y ago

I've worked with both, and BigQuery is so much easier to work with. Redshift does give you more control (to a limited extent) but it takes a lot more work and specialized knowledge to perform simple tasks well. In the article you linked, they gloss over the part that takes by far the longest when working with redshift: setting up "compression, distribution keys on large tables, and sort keys on commonly filtered columns". With BigQuery, you don't really worry about that crap, or about reloading your terabytes of data if you made a mistake somewhere in the schema. You also don't worry about vacuuming, or running out of space, or taking down the database from excessive CPU usage. Do you want to have a team of people at your company whose job is just to keep Redshift running smoothly? Or do you want another team of analysts or engineers? I could see some companies being forced to use Redshift, especially if they're using S3 a lot, and I could see people saving money in some specialized use cases, but for most people BigQuery is almost certainly going to be cheaper and faster in almost every way.

jimmytucson 8y ago
> I could see people saving money in some specialized use cases, but for most people BigQuery is almost certainly going to be cheaper and faster in almost every way That’s exactly what the TPC benchmarks are designed to show: how different appliances perform under the same diverse set of generic workloads. As of 2 years ago, they show that Redshift is faster. > With BigQuery, you don't really worry about that crap The benefit this depends on your organization’s level of expertise. If you grok sorting and distribution then you can leverage those to increase performance—but it’s not a prerequisite. For example, in Redshift, when you bulk load data into a table, if the user didn’t specify a compression scheme in the table definition, Redshift will analyze the data and find a scheme that works best, and automatically apply it to the table for you. BigQuery almost certainly does something similar. The difference is, with BigQuery, you’re not invited to take part in that discussion. And you’re charged as if the data is uncompressed. Psychologically, this is a huge relief if you don’t (want to) know how compression works but rest assured you’re paying for it somehow. To draw a tired analogy, vehicles with automatic transmission still have to shift gears. If you’re driving to the grocery store, not having to worry about that is a win. But if you’re racing stock cars, you’re definitely going to want a stick.
- deleted 8y ago
  [deleted]
- vgt 8y ago
  Compute resources tend to be significantly more expensive than storage. Our approach is two-fold: - We finance compute resources required for you to ingest data into BigQuery. With Redshift, you pay for ingest directly via compute cluster consumption (again, more expensive than storage). This also increases your complexity due to on-cluster contention of resources between ingest and query. - Separation of storage and compute gives you lots of options. With BigQuery ,you don't need to attach relatively expensive compute just for the luxury of getting more storage. Spectrum helps somewhat, but ultimately with Redshift you don't even get to pay for storage - you pay for compute/storage combos. - BigQuery's Long-Term Storage is not an archival storage tier - it's a discount on storage, with identical performance and durability characteristics. At only $0.01 per GB per month. This is likely a result of origins of the two technologies. BigQuery is Dremel, written and operated by Google since 2006. Redshift purchased source code to an on-premise fork of Postgres.

havermeyer 8y ago

From today's release notes (https://cloud.google.com/bigquery/docs/release-notes#july_25_2018 https://cloud.google.com/bigquery/docs/release-notes#july_25...): > BigQuery has achieved approximately 5x improvement in TPC-DS 10T power run geomean over the past 18 months. Outside of that, BigQuery now provides better partitioned table functionality (https://cloud.google.com/bigquery/docs/partitioned-tables https://cloud.google.com/bigquery/docs/partitioned-tables) as well as clustering (https://cloud.google.com/bigquery/docs/clustered-tables https://cloud.google.com/bigquery/docs/clustered-tables), both of which can help to reduce costs and improve performance. In terms of pricing, many large customers have reserved slots (https://cloud.google.com/bigquery/docs/slots https://cloud.google.com/bigquery/docs/slots), where you pay a fixed monthly cost in exchange for dedicated resources, similar to running a large Redshift instance. As a disclosure, I work on the BigQuery team.

vgt 8y ago

Great question. At the time we had a hard time replicating results of that blog post. Here's some food for thought. - We recently announced that we've improved our TPC-DS 10T geomean by 5x over past 18 months [0]. The nice thing is that users never had to set maintenance windows or "upgrade their clusters" to get these performance improvements. - Here's a session by our eng lead and CTO of Looker discussing various performance improvements, including querying a PB of data in ~5 seconds and paying a fraction of a penny for it.[1] - Benchmarks are important, but they miss three key scenarios. ------ Effort - how much complexity is there to achieve maximum performance. As someone else stated in this thread, we feel that we differentiate here. ------ Maintenance - when data changes, new data comes in, you run DML, what happens to this pristine benchmark of yours? What is required to maintain this performance? Again, we feel that our offering is compelling (we don't ever have messy storage state that requires a vacuum, our compute is entirely stateless so re-sorting/re-distributing data is not needed, and BigQuery ingest never affects query capacity, even at PB/day ingest scale). ------ Real-world usage - a serially-executed set of queries poorly represents what happens in reality - high volatility, high concurrency workloads. Again, BigQuery has stateless compute, separation of compute and storage, and separation of compute and state. - customers - in addition to what's been stated already, PTAL at Kabam [2] & NYT/Yahoo/BlueApron [3] for their motivations for migrating from Redshift to BigQuery That said, lots of folks really like running on Redshift, and they've extended their platform with Spectrum and other bits. I'd invite you to find out what works best for you. Competition is good for the end user! [0] https://cloud.google.com/bigquery/docs/release-notes#july_25_2018 https://cloud.google.com/bigquery/docs/release-notes#july_25... [1] https://www.youtube.com/watch?v=6Nv18xmJirs https://www.youtube.com/watch?v=6Nv18xmJirs [2] https://www.youtube.com/watch?v=6Nv18xmJirs https://www.youtube.com/watch?v=6Nv18xmJirs [3] https://www.youtube.com/watch?v=TLpfGaYWshw https://www.youtube.com/watch?v=TLpfGaYWshw

jimmytucson 8y ago
Plenty to chew on here. Thank you
- vgt 8y ago
  Please don't hesitate to reach out for follow ups