3 ms·
Most people miss this important point: the most important part of MapReduce is the shuffle step, which is a global, partitioned disk-to-disk sort. See the Flum
by dekhn 12y ago
Most people miss this important point:
the most important part of MapReduce is the shuffle step, which is a global, partitioned disk-to-disk sort. See the FlumeJava for a bit more discussion on why this is relevant.
Everything else about MapReduce is just framework to make programmer's lives easier. BTW, not having strong typing in classic MR is a major pain point. Flume goes a long way to addressing this in a practical way.
BTW, this point elucidates the importance of this classic Google Interview question: http://www.glassdoor.com/Interview/Sort-a-million-32-bit-integers-using-only-2MB-of-RAM-QTN_120936.htm http://www.glassdoor.com/Interview/Sort-a-million-32-bit-int...