3 ms·
I've never heard of Greenplum and Teradata. They are parallel databases, which is what the article is comparing MapReduce against. The basic idea is that you p
by neilc 18y ago
I've never heard of Greenplum and Teradata.
They are parallel databases, which is what the article is comparing MapReduce against. The basic idea is that you partition the data over a set of machines, and similarly divide and parallelize query processing.
MR is basically a specialized parallel DB that executes a single query of the form:
SELECT map(...), reduce_agg(...)
FROM ...
GROUP BY map(...)
Again, MR's architecture differs in various ways from the design of a typical parallel DB, but they are certainly comparable.
It doesn't make any sense, for example, to talk about loading data into MapReduce
MR input data is typically stored on a distributed filesystem (e.g. HDFS, GFS). Loading the data onto that filesystem is analogous to loading the data into a parallel DB; in both cases, the input data is partitioned over a set of nodes. For good performance, you want a copy of the data to reside close to where the subsequent computation is going to take place. But certainly MR and DBs differ in this respect: a DB typically assumes it "owns" the data, whereas MR is more amenable to processing data stored in an arbitrary external format. This difference is not fundamental, though.
I strongly doubt relational databases are used for precisely the same (or even somewhat similar) tasks.
You would be mistaken, then. If you want to do analysis on large volumes of data using a cluster of machines, both parallel DBs and MR are viable choices. The whole point of the paper is to take a set of queries, execute them using both Hadoop and parallel DBs, and compare the resulting performance and user experience.