3 ms·
This isn't clear in the blog post; I'll try to clarify. The buckets correspond to hash token ranges. For example, the database takes one row from orders_2013,
by ozgune 11y ago
This isn't clear in the blog post; I'll try to clarify.
The buckets correspond to hash token ranges. For example, the database takes one row from orders_2013, hashes the customer_id value, and gets a hashed value of 1.5B. It then puts it into the second bucket orders_2013_[1B-2B[.
After the distributed shuffle, you end up with orders_cust_id[1B-2B[ on machine 2. So all orders that belong to a particular / disjoint set of customers live on one machine. You can then push down the count(distinct customer_id) to each node, and then add up the results.
We use hash token ranges in pg_shard. I'll include this clarification to the blog post.