4 ms·
Meet Big Data equivalent of the LAMP Stack
- yarapavan 16y agoSummary: The Big Data Stack comprises of: * Hadoop Distributed File System (HDFS) for storage * MapReduce for distributed processing of large data sets on compute clusters * HBase for fast read/write access to tabular data * Hive for SQL-like queries on large data sets as well as a columnar storage layout using RCFile * Flume for log file and streaming data collection, along with Sqoop for database imports * JDBC and ODBC drivers to allow tools written for relational databases to access data stored in Hive * Hue for user interfaces * Pig for dataflow and parallel computations * Oozie for workflow * Avro for serialization * Zookeeper for coordinated service for distributed applications
- Silhouette 16y agoI think that explains a lot about why not many people seem to be actually using alternatives to LAMP (as opposed to talking about them on forums like this one). LAMP is simple: OS as foundation, web server, database server, programming language to join them up. I wonder how many people reading this article know what all of the above technologies do without looking them up...
- michaels0620 16y agoThis stack looks to be described at a much lower level then LAMP which (as you point out) makes it more confusing. At the very least, they could consolidate HDFS, Hue, HBase, and Hive into just "Hadoop".
- adnam 16y agoHMHHFJHPOAZ doesn't quite have the same ring to it, although I look forward to listing it on my CV.
- auxbuss 16y agoShould net you a well-paid job, presuming that you have the usual ten years experience in it.
- gojomo 16y agoExcellent point. So I propose instead "HART" -- "Hadoop And Related Tools". The HART stack.
- ratsbane 16y agoUpvoted, seconded, and added to my CV. I was just thinking, in reading that article, that one of the biggest obstacles to selling the HART stack to management types are the names. Can you really, with a straight face, imagine standing before a room full of grey-haired VPs and saying "yes, we want to implement this with Hadoop using Hive, Flume, Hue, Pig, Oozie, Avro, and Zookeeper." And then explain that "Hadoop" is named after a teddy bear.
- anamax 16y ago> And then explain that "Hadoop" is named after a teddy bear. Hadoop is named after a stuffed elephant.
- die_sekte 16y agoWhich is German for "hard". I'm not sure if that association is good.
- buro9 16y agoNice... so can someone knock up a load of chef scripts and make installing and configuring this as simple as it should be? What would also be good would be a summary diagram of how this looks as an architecture. Just how should it be implemented in small/medium/large configs, and what is the definition of S/M/L in terms of data size/complexity? Basically... if I want to use it, where's the simple intro and the quick start? That's the advantage LAMP has, you can be up and running in 30 minutes.
- rbranson 16y agoSo this stack will curse us all with having to fix awful programming mistakes for decades?
- d2viant 16y agoCould you be more specific?
- earl 16y agosweet christ. that reads like fucking trendy hadoop word soup spat out to get suckers to hire this firm for $500 an hour. Also, how does he think that installing this crap is ever going to be like installing excel, until you're installing and configuring excel into your custom datacenter topography