4 ms·
Yes, stuff that splits up, shuffles, and combines the result is exactly what I mean. That’s MapReduce. It’s just that e.g. in Dremel/BigQuery the shuffle step h
by ein0p 3y ago
Yes, stuff that splits up, shuffles, and combines the result is exactly what I mean. That’s MapReduce. It’s just that e.g. in Dremel/BigQuery the shuffle step happens via a special in-memory distributed system, but the fundamental idea is the same: do what you can on the node, partition/shuffle, reduce on the receiving nodes, sometimes more than once.