5 ms·

I’m one of the authors, happy to answer questions!

by georgewfraser 8y ago

kwillets 8y ago
No Vertica?
- tomnipotent 8y ago
  Does Vertica have a SaaS offering?
  - kwillets 8y ago
    By the hour: https://aws.amazon.com/marketplace/pp/B079VR7CD7 https://aws.amazon.com/marketplace/pp/B079VR7CD7
    - secondrow 8y ago
      Presto Starburst distro (which has been upgraded to 203e since the benchmark) - also with hourly pricing https://aws.amazon.com/marketplace/pp/B07DKV5659?qid=1536609469418 https://aws.amazon.com/marketplace/pp/B07DKV5659?qid=1536609...
vgt 8y ago
Great post! A couple of thoughts: 1. It'd be interesting to see these benchmarks "under real-world adversity". For example, as data is ingested into the DW, as data shapes change, as concurrency of workloads varies. 2. It'd be great to measure "effort" to reach and maintain this performance. DW vendors are trained on optimizing for static TPC benchmarks with concurrency of 1, which are useful, but can miss the boat on what users actually experience. (BQ PM)
- georgewfraser 8y ago
  We intend to keep iterating on this! If you have a concrete idea, it would be fantastic if you opened an issue or even a PR in our github repo: https://github.com/fivetran/benchmark https://github.com/fivetran/benchmark
- massaman_yams 8y ago
  Thanks for the insight! Maybe it was just a brain fart, but I had to look up BQ (BigQuery) just now - the acronym might not be as well-known outside of Google.
  - vgt 8y ago
    noted :)
djhworld 8y ago
I'd like to understand more details about your Presto testing. The post mentioned ORC files, but doesn't detail how big they are, how many, is the dataset partitioned? Is the data sorted in any particular way?
massaman_yams 8y ago
This is fantastic, thanks so much for putting this together. In the past I've relied a bit too much on benchmarks here [0] for assessment of warehousing solutions, but it's pretty clearly skewed toward certain query patterns due to operating on a single table (no joins) and limited aggregation. Using TPC with a wider variety of queries and evaluating on dimensions other than performance provides much better perspective. [0] http://tech.marksblogg.com/benchmarks.html http://tech.marksblogg.com/benchmarks.html

I’m one of the authors, happy to answer questions!

kwillets 8y ago

No Vertica?

tomnipotent 8y ago
Does Vertica have a SaaS offering?
- kwillets 8y ago
  By the hour: https://aws.amazon.com/marketplace/pp/B079VR7CD7 https://aws.amazon.com/marketplace/pp/B079VR7CD7
  - secondrow 8y ago
    Presto Starburst distro (which has been upgraded to 203e since the benchmark) - also with hourly pricing https://aws.amazon.com/marketplace/pp/B07DKV5659?qid=1536609469418 https://aws.amazon.com/marketplace/pp/B07DKV5659?qid=1536609...

vgt 8y ago

Great post! A couple of thoughts: 1. It'd be interesting to see these benchmarks "under real-world adversity". For example, as data is ingested into the DW, as data shapes change, as concurrency of workloads varies. 2. It'd be great to measure "effort" to reach and maintain this performance. DW vendors are trained on optimizing for static TPC benchmarks with concurrency of 1, which are useful, but can miss the boat on what users actually experience. (BQ PM)

georgewfraser 8y ago
We intend to keep iterating on this! If you have a concrete idea, it would be fantastic if you opened an issue or even a PR in our github repo: https://github.com/fivetran/benchmark https://github.com/fivetran/benchmark
massaman_yams 8y ago
Thanks for the insight! Maybe it was just a brain fart, but I had to look up BQ (BigQuery) just now - the acronym might not be as well-known outside of Google.
- vgt 8y ago
  noted :)

djhworld 8y ago

I'd like to understand more details about your Presto testing. The post mentioned ORC files, but doesn't detail how big they are, how many, is the dataset partitioned? Is the data sorted in any particular way?

massaman_yams 8y ago

This is fantastic, thanks so much for putting this together. In the past I've relied a bit too much on benchmarks here [0] for assessment of warehousing solutions, but it's pretty clearly skewed toward certain query patterns due to operating on a single table (no joins) and limited aggregation. Using TPC with a wider variety of queries and evaluating on dimensions other than performance provides much better perspective. [0] http://tech.marksblogg.com/benchmarks.html http://tech.marksblogg.com/benchmarks.html