4 ms·
haven't come across this idea before, interesting - will definitely have to give it some more thought. our 'distributed joins', so to speak, run through our dis
by irfansharif 9y ago
haven't come across this idea before, interesting - will definitely have to give it some more thought. our 'distributed joins', so to speak, run through our distributed query execution model (distsql) setting up incremental 'stages' of computation with the results pipelined and plumbed through individual computes. viewing it through this model our implementation more closely resembles the Grace Hash Join[1] algorithm. you might be interested in the PR[2] that landed this changeset, there's a cool visualization in one of the comments[3] showing the query execution plan.
[1]: https://en.wikipedia.org/wiki/Hash_join#Grace_hash_join https://en.wikipedia.org/wiki/Hash_join#Grace_hash_join
[2]: https://github.com/cockroachdb/cockroach/pull/12221 https://github.com/cockroachdb/cockroach/pull/12221
[3]: https://github.com/cockroachdb/cockroach/pull/12221#issuecomment-267084941 https://github.com/cockroachdb/cockroach/pull/12221#issuecom...
- ww520 9y agoThe Grace Hash Join approach ships the entire joining key set across network. Even if each node just get one partition of it, the aggregate network traffic is the entire set. For small table, it's fine. Large table is going to really tax the network.