3 ms·
To me, Hadoop was primarily designed around large-scale semi-structured data like logs; this plays to its strengths of scale and schema-on-read. If I'm not mis
by wenc 8y ago
To me, Hadoop was primarily designed around large-scale semi-structured data like logs; this plays to its strengths of scale and schema-on-read.
If I'm not mistaken, Hadoop was created at Google to handle weblogs (side note: in 2014, Google announced that Hadoop was no longer being used internally)
Enterprise data however is primarily structured and relational, which is more suited to handling in a database-like system. Hadoop was never designed for this use case. Scalable cloud databases like Redshift, Aurora etc. always seemed to be a better fit. Cloudera created technology like Impala and Kudu to address this but not sure about the uptake there.
- atombender 8y agoNo, Hadoop was started at Yahoo! by Doug Cutting, the original author of Lucene. Hadoop was based on Google's published papers on the Google File System (GFS) and MapReduce. The later project, HBase, was a direct carbon copy of Google's BigTable paper. By the time Hadoop reached maturity, Google had mostly moved onto newer technologies such as Colossus and Megastore, and mostly doesn't use MapReduce anymore. To my knowledge, Google has never used Hadoop internally, although you can lease a hosted version of Hadoop on Google Cloud Platform.
- wenc 8y agoYou're quite right. It was GFS and MapReduce that was invented by Google, and Doug Cutting based it on those technologies.