5 ms·
Anyone got a succinct explanation for why ClickHouse is doing so well in this kind of benchmark? What are the key tricks they're using to get great analytical p
by simonw 4y ago
Anyone got a succinct explanation for why ClickHouse is doing so well in this kind of benchmark? What are the key tricks they're using to get great analytical performance, and why haven't those same tricks been adapted by other competing systems?
- zX41ZdbW 4y agoI see that other systems often forgot many small details. Imagine you are implementing COUNT(DISTINCT) function. And imagine it is used in GROUP BY something, like GROUP BY region_id. And on realistic datasets, you will have a large number of different region_id and something like power law distribution of their frequency: https://en.wikipedia.org/wiki/Zipf%27s_law https://en.wikipedia.org/wiki/Zipf%27s_law Then you will expect that most of the values of COUNT(DISTINCT) will be small. And the natural optimization will be - using a linear array before using a hash table. Or: place smaller hash tables in something like a memory pool. This is about a ton of optimizations on distributions of real datasets.
- drchaim 4y agoI would say deep optimizations in every aspect of the data cycle. From storing to querying data is heavily optimized. I recall an online meetup where a 10ms gain after a patch was huge celebrated. That kind of things compound over and over and make a really fast data platform for that use cases.