4 ms·

Relational Database Experts Jump The MapReduce Shark

22 points by iamelgringo 19y ago

boucher 19y ago
This is a good refutation of the linked article. The authors of the original article definitely seem to misunderstand both the purpose and the usefulness of MapReduce. The claim that MapReduce is a poor implementation, for example, is particularly silly. Most large scale relational databases rely on very powerful centralized machines, and replicas of those machines. MapReduce, by contrast, relies on a large number of commodity systems. The decentralized nature makes hardware failure much less likely to cause problems. The system knows how to dynamically reallocate tasks, and new nodes can also be added while the system is running.
joshwa 19y ago
I think perhaps the original authors were really thinking of BigTable: http://en.wikipedia.org/wiki/Bigtable http://en.wikipedia.org/wiki/Bigtable
- neilc 19y ago
  What makes you think that? I think DeWitt and Stonebraker can distinguish between BigTable and MapReduce (indeed, their blog post mentions BigTable and Hadoop as well).
tlrobinson 19y ago
Clearly these guys need to stick to commentary on RDBMSs, which MapReduce most definitely is not. MapReduce enables Google to do it's thing, so I wouldn't call it a "major step backwards".
- neilc 19y ago
  The fact that Google use MapReduce has no bearing on whether it is a "major step backwards" to the database community. The original authors essentially argue that MapReduce ignores 40 years of database technology, and that it is the worse for it; whether MapReduce is essential at Google isn't the point.
  - Kaizyn 19y ago
    You're absolutely correct here. Wouldn't most users consider the fact that it ignores database research and history a feature of MapReduce?
cridal 19y ago
Why don't those guys descend from their ivory tower and build something novel, interesting and useful? I'm sure the solution to today's exponential data explosion is building one giant relational database inside of Mount Kilimanjaro...
- Shooter 19y ago
  RE: "ivory-tower", "something useful" http://en.wikipedia.org/wiki/Michael_Stonebraker http://en.wikipedia.org/wiki/Michael_Stonebraker
  - jpbf 19y ago
    Sure the guy is academically merited. But it's not the first time he compares apples to pears. http://dlweinreb.wordpress.com/2007/12/31/object-oriented-database-management-systems-succeeded/ http://dlweinreb.wordpress.com/2007/12/31/object-oriented-da... This everything must be a RDBMS viewpoint seem rather rigid.
    - Shooter 19y ago
      So...you read the Wikipedia link and the main thing you got out of it is that the guy is "academically merited"?! The part about him founding five DBMS companies and being the CTO of one of the largest DBMS companies in the world doesn't suggest that maybe he might have some practical experience? The fact that he made some major contributions to the database field somehow leads you to believe he is now confused and talking complete nonsense? No? Yes? Okay...I guess he is just an ivory tower egghead. My bad. I've read the Weinreb post...and I agree with Dan's points, but they're really just disagreeing about the definition of success. I've tried to use object databases in industry, and I can assure you that they ARE considered failures by most people. (I'm not saying their bad reputation is entirely deserved, however...) By the way, Stonebraker saying that object databases are a 'failure' is not the same as him saying that RDBMS should be used for everything. Apples and Pears ;-) Actually, Stonebraker is one of the most vocal proponents of using non-RDBMS technologies when appropriate. See the "One Size Fits All" paper and its follow-up. I'm not saying I agree with everything (or even anything) that Stonebraker writes, but I think it is a mistake to discount his opinion out of hand because you erroneously think he is just some confused egghead who isn't up on the latest technologies and is clinging to the RDBMS technology he is comfortable with. That does you both a disservice. People tend to write blog posts that argue one side or the other...the 'truth' often lies somewhere between the two sides. P.S. If my reply is unnecessarily nasty, it is partly due to the fact that I have insomnia and I'm therefore pissed off (it's 4:30am here and I have to leave on a cross-country flight in two hours.)
bayareaguy 19y ago
You have to read the original article very closely or you'll miss their point: in terms of distributed database research, MapReduce is a step backwards. So what else is new? Things interesting to database researchers often take two decades to become mainstream. To be interesting to a database researcher you need to think way beyond MapReduce. Stuff like distributing queries over 10^12 RFID sensor nodes each storing different data in a different way, optimizing in real time and adapting to arbitrary breakdowns in the communications network.
marcus 19y ago
The beauty of MapReduce is in its elegant simplicity of use (once properly implemented) think of Unix shell commands like grep, sort, cut - simple functional tools (as in functional languages - zero side effects), which do only one task but do it amazingly well, Google just added a new type of screwdriver to the toolbox.
g00dn3ss 19y ago
This title makes no sense. Was Fonzie jumping over a more popular show disguised as a shark? Let's mix some metaphors!
- schorndorfer 19y ago
  I absolutely agree -- the phrase"jump the shark" has now jumped the shark. Seems to be egregiously mis-used and over-used especially by tech bloggers.

22 points by iamelgringo 19y ago

This is a good refutation of the linked article. The authors of the original article definitely seem to misunderstand both the purpose and the usefulness of MapReduce. The claim that MapReduce is a poor implementation, for example, is particularly silly. Most large scale relational databases rely on very powerful centralized machines, and replicas of those machines. MapReduce, by contrast, relies on a large number of commodity systems. The decentralized nature makes hardware failure much less likely to cause problems. The system knows how to dynamically reallocate tasks, and new nodes can also be added while the system is running.

joshwa 19y ago

I think perhaps the original authors were really thinking of BigTable: http://en.wikipedia.org/wiki/Bigtable http://en.wikipedia.org/wiki/Bigtable

neilc 19y ago
What makes you think that? I think DeWitt and Stonebraker can distinguish between BigTable and MapReduce (indeed, their blog post mentions BigTable and Hadoop as well).

tlrobinson 19y ago

Clearly these guys need to stick to commentary on RDBMSs, which MapReduce most definitely is not. MapReduce enables Google to do it's thing, so I wouldn't call it a "major step backwards".

neilc 19y ago
The fact that Google use MapReduce has no bearing on whether it is a "major step backwards" to the database community. The original authors essentially argue that MapReduce ignores 40 years of database technology, and that it is the worse for it; whether MapReduce is essential at Google isn't the point.
- Kaizyn 19y ago
  You're absolutely correct here. Wouldn't most users consider the fact that it ignores database research and history a feature of MapReduce?

cridal 19y ago

Why don't those guys descend from their ivory tower and build something novel, interesting and useful? I'm sure the solution to today's exponential data explosion is building one giant relational database inside of Mount Kilimanjaro...

Shooter 19y ago
RE: "ivory-tower", "something useful" http://en.wikipedia.org/wiki/Michael_Stonebraker http://en.wikipedia.org/wiki/Michael_Stonebraker
- jpbf 19y ago
  Sure the guy is academically merited. But it's not the first time he compares apples to pears. http://dlweinreb.wordpress.com/2007/12/31/object-oriented-database-management-systems-succeeded/ http://dlweinreb.wordpress.com/2007/12/31/object-oriented-da... This everything must be a RDBMS viewpoint seem rather rigid.
  - Shooter 19y ago
    So...you read the Wikipedia link and the main thing you got out of it is that the guy is "academically merited"?! The part about him founding five DBMS companies and being the CTO of one of the largest DBMS companies in the world doesn't suggest that maybe he might have some practical experience? The fact that he made some major contributions to the database field somehow leads you to believe he is now confused and talking complete nonsense? No? Yes? Okay...I guess he is just an ivory tower egghead. My bad. I've read the Weinreb post...and I agree with Dan's points, but they're really just disagreeing about the definition of success. I've tried to use object databases in industry, and I can assure you that they ARE considered failures by most people. (I'm not saying their bad reputation is entirely deserved, however...) By the way, Stonebraker saying that object databases are a 'failure' is not the same as him saying that RDBMS should be used for everything. Apples and Pears ;-) Actually, Stonebraker is one of the most vocal proponents of using non-RDBMS technologies when appropriate. See the "One Size Fits All" paper and its follow-up. I'm not saying I agree with everything (or even anything) that Stonebraker writes, but I think it is a mistake to discount his opinion out of hand because you erroneously think he is just some confused egghead who isn't up on the latest technologies and is clinging to the RDBMS technology he is comfortable with. That does you both a disservice. People tend to write blog posts that argue one side or the other...the 'truth' often lies somewhere between the two sides. P.S. If my reply is unnecessarily nasty, it is partly due to the fact that I have insomnia and I'm therefore pissed off (it's 4:30am here and I have to leave on a cross-country flight in two hours.)

bayareaguy 19y ago

You have to read the original article very closely or you'll miss their point: in terms of distributed database research, MapReduce is a step backwards. So what else is new? Things interesting to database researchers often take two decades to become mainstream. To be interesting to a database researcher you need to think way beyond MapReduce. Stuff like distributing queries over 10^12 RFID sensor nodes each storing different data in a different way, optimizing in real time and adapting to arbitrary breakdowns in the communications network.

marcus 19y ago

The beauty of MapReduce is in its elegant simplicity of use (once properly implemented) think of Unix shell commands like grep, sort, cut - simple functional tools (as in functional languages - zero side effects), which do only one task but do it amazingly well, Google just added a new type of screwdriver to the toolbox.

g00dn3ss 19y ago

This title makes no sense. Was Fonzie jumping over a more popular show disguised as a shark? Let's mix some metaphors!

schorndorfer 19y ago
I absolutely agree -- the phrase"jump the shark" has now jumped the shark. Seems to be egregiously mis-used and over-used especially by tech bloggers.