3 ms·
By exporting this data first into HDFS, I can now run Map/Reduce jobs more quickly and not hang up DB connections Right; the typical configuration is to have o
by neilc 17y ago
By exporting this data first into HDFS, I can now run Map/Reduce jobs more quickly and not hang up DB connections
Right; the typical configuration is to have one database that does transaction processing, and another separate database (a "data warehouse") that collects data from multiple operational DBs and runs analytic queries over it. Hadoop is basically just an alternative analytic query processor in this configuration.
Hadoop becomes incrementally more useful as the size of the files increase
I think the same would apply to parallel DBs: you are basically talking about just partitioning the data over the storage nodes, which is a common feature in both systems.