4 ms·
> Of course the way to implement GROUP BY is with a hash table but little tricks make all the difference and a lot of love went into the clickhouse implementati
by jfim 7y ago
> Of course the way to implement GROUP BY is with a hash table but little tricks make all the difference and a lot of love went into the clickhouse implementation.
Actually, for low cardinality columns, _not_ using a hash table will speed up things.
For example, a dictionary-encoded column for states might have a few values in its dictionary (1 => CA, 2 => FL, 3 => NY) and the data looks like an array of numbers (eg. [1, 1, 1, 2, 1, 3, 3, 3, 1, 3, 2]). The fastest way to aggregate is to actually use an array, as the dictionary index conveniently maps to an array index.
Then, when it comes to merging several of those arrays, they're turned into hash tables.
Combine enough of those optimizations and proper data layouts, and you end up with several orders of magnitude of performance differences between engines.