4 ms·
map/reduce is a very simple, very general idea. It is also Google's and Hadoop's specific implementations with lots of bells and whistles. It's also everythin
by adamtj 16y ago
map/reduce is a very simple, very general idea. It is also Google's and Hadoop's specific implementations with lots of bells and whistles. It's also everything in between. This implementation has more than the minimum two necessary features. If you are arguing that it isn't a true map/reduce implementation, then I would argue that you aren't a true software developer. http://en.wikipedia.org/wiki/No_true_scotsman http://en.wikipedia.org/wiki/No_true_scotsman
- btilly 16y agoThe idea of MapReduce was introduced to the world in the paper http://labs.google.com/papers/mapreduce.html http://labs.google.com/papers/mapreduce.html. The abstract starts off with MapReduce is a programming model and an associated implementation for processing and generating large data sets. Users specify a map function that processes a key/value pair to generate a set of intermediate key/value pairs, and a reduce function that merges all intermediate values associated with the same intermediate key. Many real world tasks are expressible in this model, as shown in the paper. If you lack that feature set, then you can't solve most of the problems that MapReduce is actually used for in practice. Conversely if you have that feature set, then you can, though not necessarily with good reliability or at optimal speed. Therefore that description is what I take as the canonical description of what it means to have implemented MapReduce. As for your arguing that I am not a true software developer, guilty as charged. I've been joking for years that I am merely a profitably displaced mathematician, and my current job role is a weird cross between software development and system administration.