3 ms·

Author of that blogpost here. > less "code branches" than duckdb, which may or may not translate to faster performance. In that case it was about 2.5x faster

Author of that blogpost here. > less "code branches" than duckdb, which may or may not translate to faster performance. In that case it was about 2.5x faster than DuckDB end to end, so a bit less than the difference in branches. If you want to see some independent benchmarks on Umbra, our underlying technology, its currently first place on Clickbench [1]. You can compare against duckdb there as well. [1] https://benchmark.clickhouse.com/ https://benchmark.clickhouse.com/

riku_iki 2y ago
clickbench is a toy benchmark: small dataset, very specific queries. Benchmarking full tcp-h (not just one query like in your post) on sizable dataset (few TBs) would be very good close to real world scenario, but vendors usually avoid this.
- pfent 2y ago
  There are TPC-H numbers in another post: https://cedardb.com/blog/simple_efficient_hash_tables/ https://cedardb.com/blog/simple_efficient_hash_tables/
  - riku_iki 2y ago
    its great starting insight, but again its small dataset (100GB) which almost fits memory, and I think many details are missing (for example clickbench publishes all configs and queries, and more detailed report, so vendors can reproduce/optimize/dispute them).
    - refset 2y ago
      > small dataset (100GB) What counts as large or small definitely varies a lot depending on the context of the conversation/analysis. MotherDuck's "Big Data is Dead" post [0] sticks in mind: > The general feedback we got talking to folks in the industry was that 100 GB was the right order of magnitude for a data warehouse. This is where we focused a lot of our efforts in benchmarking. Another point of reference is [1] > [...] Umbra achieves unprecedentedly low query latencies. On small data sets, it is even faster than interpreter engines like DuckDB > TPC-H Small Dataset = 866k tuples, sf 0.1 [0] https://motherduck.com/blog/big-data-is-dead/ https://motherduck.com/blog/big-data-is-dead/ [1] https://db.in.tum.de/~kersten/Tidy%20Tuples%20and%20Flying%20Start%20Fast%20Compilation%20and%20Fast%20Execution%20of%20Relational%20Queries%20in%20Umbra.pdf?lang=de https://db.in.tum.de/~kersten/Tidy%20Tuples%20and%20Flying%2...
      - riku_iki 2y ago
        > The general feedback we got talking to folks in the industry was that 100 GB and then user discovers that DuckDB is plagued with OOMs and dramatic performance degradations when his data is slightly larger than memory.

riku_iki 2y ago

clickbench is a toy benchmark: small dataset, very specific queries. Benchmarking full tcp-h (not just one query like in your post) on sizable dataset (few TBs) would be very good close to real world scenario, but vendors usually avoid this.

pfent 2y ago
There are TPC-H numbers in another post: https://cedardb.com/blog/simple_efficient_hash_tables/ https://cedardb.com/blog/simple_efficient_hash_tables/
- riku_iki 2y ago
  its great starting insight, but again its small dataset (100GB) which almost fits memory, and I think many details are missing (for example clickbench publishes all configs and queries, and more detailed report, so vendors can reproduce/optimize/dispute them).
  - refset 2y ago
    > small dataset (100GB) What counts as large or small definitely varies a lot depending on the context of the conversation/analysis. MotherDuck's "Big Data is Dead" post [0] sticks in mind: > The general feedback we got talking to folks in the industry was that 100 GB was the right order of magnitude for a data warehouse. This is where we focused a lot of our efforts in benchmarking. Another point of reference is [1] > [...] Umbra achieves unprecedentedly low query latencies. On small data sets, it is even faster than interpreter engines like DuckDB > TPC-H Small Dataset = 866k tuples, sf 0.1 [0] https://motherduck.com/blog/big-data-is-dead/ https://motherduck.com/blog/big-data-is-dead/ [1] https://db.in.tum.de/~kersten/Tidy%20Tuples%20and%20Flying%20Start%20Fast%20Compilation%20and%20Fast%20Execution%20of%20Relational%20Queries%20in%20Umbra.pdf?lang=de https://db.in.tum.de/~kersten/Tidy%20Tuples%20and%20Flying%2...
    - riku_iki 2y ago
      > The general feedback we got talking to folks in the industry was that 100 GB and then user discovers that DuckDB is plagued with OOMs and dramatic performance degradations when his data is slightly larger than memory.