3 ms·
To be clear, many of the comments are from Bob Muglia, CEO of Snowflake Computing which is a cloud based columnar data warehouse solution. Presumably, much eas
by sixdimensional 10y ago
To be clear, many of the comments are from Bob Muglia, CEO of Snowflake Computing which is a cloud based columnar data warehouse solution. Presumably, much easier to implement than Hadoop, so you have to take those comments with a grain of salt.
Also, those entities who the article claims have a grasp on Hadoop are some of the main people who helped develop it - lest we forget that the early work on Hadoop came out of Yahoo. "Web-scale" companies by their very nature have to own this technology, because almost nobody else on earth did what they have done on that scale (except maybe governments and scientists).
However, I agree the article is right on. As someone who has done a lot of work with different Hadoop distributions (Cloudera, Hortonworks and MapR to name a few), I can say that compared to the performance and ease of use of more traditional data platforms, Hadoop is a different and more complex creature. It has evolved quickly in some regards and slowly in others. It's way easier to get a Cloudera system up and running today (for example) than it was in the past.
"Hadoop" by itself is a misnomer, it's not like it's one technology. HDFS, MapReduce (and MR2), Hive, HBase, Spark, Pig, Flume, etc. etc. - it's an ecosystem. It's also a mashup of many things depending on what distro you're talking about - too many things - a "data hub", an "operational data store", a "pre-data warehouse staging", an ETL platform, a "search" platform, a data governance platform, a data archiving solution, a real-time data query / streaming data processing etc. etc. etc.
I think Hadoop did one important thing to our overall awareness, and that was to think of a "data platform", "data processing", "data architecture" and "data infrastructure" as a major, and "big" strategic part of an organization. I think the rise of "big data" thinking along with the rise of the "chief data officer", "data scientists" and "data engineers" and the like all goes together - a shift in thinking we made where we realized we could do more with more data (for better or worse).
There are many new and old technologies that help people implement the data infrastructure in their organizations - not just Hadoop. Hadoop just happened to be a major player at the epicenter at this (and open source as well), and if anything, has spurred technological advancement in databases, NoSQL and NewSQL and OldSQL alike, ETL, data mining, AI/machine learning, etc. So it has played its role, whether it specifically stays relevant or not as we perceive it.