3 ms·
Opinions are my own. Some info on flume: https://research.google/pubs/pub35650/ https://research.google/pubs/pub35650/ To quote from there: "MapReduce and sim
by gravypod 3y ago
Opinions are my own.
Some info on flume: https://research.google/pubs/pub35650/ https://research.google/pubs/pub35650/
To quote from there: "MapReduce and similar systems significantly ease the task of writing data-parallel code. However, many real-world computations require a pipeline of MapReduces, and programming and managing such pipelines can be difficult."
- itissid 3y agoAren't hash joins done in RDBMS just like a general application of map-reduce? In left joins The big table is hashed on the join key value and sent to N machines and the little table is just everywhere. IIUC this is how any OLAP/Bigdata frameworks think while doing massive joins or partitioning to reduce data later, they just have to deal with additional issues like locality of partition to computation target. So map reduce is in the DNA of many data computation flows instead of a thing in off itself.