http://www.computerworld.com/s/article/9142406/Big_three_database_vendors_diverge_on_Hadoop http://www.computerworld.com/s/article/9142406/Big_three_dat... "We

http://www.computerworld.com/s/article/9142406/Big_three_database_vendors_diverge_on_Hadoop http://www.computerworld.com/s/article/9142406/Big_three_dat... "We'd never bring Hadoop code into one of our products," said Microsoft's David J. DeWitt. DeWitt is an academic expert in parallel SQL databases. DeWitt says that in MapReduce "schema is buried" and furthermore, "the programmer must discover the structure by an examination of the code. Not only is this a very tedious exercise, but also the programmer must find the source code for the application." Ever heard of Hive ? http://wiki.apache.org/hadoop/Hive/GettingStarted#SQL_Operations http://wiki.apache.org/hadoop/Hive/GettingStarted#SQL_Operat... But he does make one important point - "whether a DBMS should be written: a. By stating what you want ( Relational DBMS ) b. By presenting an algorithm for data access ( Codasyl, MapReduce) Well, mathematically speaking, 'a' wins hands down since you've normalized the data & have "no garbage in the data set" ( DeWitt's terminology ) . However, once you have a fast-enough access, potentially infinite memory to handle thousands of columns & millions of rows, then there's no reason not to atleast try to do a Codasyl. In that respect, Codasyl is like say a bubblesort. If you're going to be sorting atmost 10 elements a million times in your application, you would be better off with a quick and dirty bubblesort which actually performs faster in this particular case than a correctly written MergeSort which will do a lot better if you have a million elements, but will perform poorly if you have just 10 elements ( O(1n^2) = 100, O(510*ln(10)) = 115 ). When I first learnt DBMSs in school, the Professor actually made this very point - "someday we'll attempt a Codasyl, just not right now." Well, that day has come.