4 ms·
Summary: The Big Data Stack comprises of: * Hadoop Distributed File System (HDFS) for storage * MapReduce for distributed processing of large data sets on co
by yarapavan 16y ago
Summary:
The Big Data Stack comprises of:
* Hadoop Distributed File System (HDFS) for storage
* MapReduce for distributed processing of large data sets on compute clusters
* HBase for fast read/write access to tabular data
* Hive for SQL-like queries on large data sets as well as a columnar storage layout using RCFile
* Flume for log file and streaming data collection, along with Sqoop for database imports
* JDBC and ODBC drivers to allow tools written for relational databases to access data stored in Hive
* Hue for user interfaces
* Pig for dataflow and parallel computations
* Oozie for workflow
* Avro for serialization
* Zookeeper for coordinated service for distributed applications
- Silhouette 16y agoI think that explains a lot about why not many people seem to be actually using alternatives to LAMP (as opposed to talking about them on forums like this one). LAMP is simple: OS as foundation, web server, database server, programming language to join them up. I wonder how many people reading this article know what all of the above technologies do without looking them up...
- michaels0620 16y agoThis stack looks to be described at a much lower level then LAMP which (as you point out) makes it more confusing. At the very least, they could consolidate HDFS, Hue, HBase, and Hive into just "Hadoop".
- adnam 16y agoHMHHFJHPOAZ doesn't quite have the same ring to it, although I look forward to listing it on my CV.
- auxbuss 16y agoShould net you a well-paid job, presuming that you have the usual ten years experience in it.
- gojomo 16y agoExcellent point. So I propose instead "HART" -- "Hadoop And Related Tools". The HART stack.
- ratsbane 16y agoUpvoted, seconded, and added to my CV. I was just thinking, in reading that article, that one of the biggest obstacles to selling the HART stack to management types are the names. Can you really, with a straight face, imagine standing before a room full of grey-haired VPs and saying "yes, we want to implement this with Hadoop using Hive, Flume, Hue, Pig, Oozie, Avro, and Zookeeper." And then explain that "Hadoop" is named after a teddy bear.
- anamax 16y ago> And then explain that "Hadoop" is named after a teddy bear. Hadoop is named after a stuffed elephant.
- die_sekte 16y agoWhich is German for "hard". I'm not sure if that association is good.
- gojomo 16y agoHard as in 'difficult' or hard like a diamond? (Are they the same in German like English?) 'Lamp' means 'light source', a vaguely positive but not highly relevant association. Being a homophone for 'heart' in English creates a vaguely positive association as well.
- die_sekte 16y agoBoth. Although slightly tipped towards diamond!hard.
- buro9 16y agoNice... so can someone knock up a load of chef scripts and make installing and configuring this as simple as it should be? What would also be good would be a summary diagram of how this looks as an architecture. Just how should it be implemented in small/medium/large configs, and what is the definition of S/M/L in terms of data size/complexity? Basically... if I want to use it, where's the simple intro and the quick start? That's the advantage LAMP has, you can be up and running in 30 minutes.