4 ms·
Nope and nope (unless you're generalizing MapReduce to "stuff that splits up work and combines the results) ;) Scaled, distributed joining has a small army of
by rossjudson 3y ago
Nope and nope (unless you're generalizing MapReduce to "stuff that splits up work and combines the results) ;)
Scaled, distributed joining has a small army of techniques working hard these days.
- ein0p 3y agoYes, stuff that splits up, shuffles, and combines the result is exactly what I mean. That’s MapReduce. It’s just that e.g. in Dremel/BigQuery the shuffle step happens via a special in-memory distributed system, but the fundamental idea is the same: do what you can on the node, partition/shuffle, reduce on the receiving nodes, sometimes more than once.