4 ms·
I started with Hive and worked backwards. It gives you a nice SQL interface and allows you to do M/R operations on CSV files easily. Once you get the hang of it
by heartsucker 10y ago
I started with Hive and worked backwards. It gives you a nice SQL interface and allows you to do M/R operations on CSV files easily. Once you get the hang of it, going back towards raw M/R or even something like Cascading/Scalding might be less of a shock.
If you know Cassandra or other NoSQL, you can try your hand at Hbase. To do anything with it beyond adding or removing data from a key, you'll need to write an application of some sort. Cataloging tweets is a decently simple exercise.
In my work, the only time I accessed the HDFS directly was doing a put/delete of a flat file CSV that I was going to load into Hive. I'm not saying there's not use cases for using HDFS, just that in the set ups I've used, I've never seen it.
- hubatrix 10y agoBut my as its part of my academics, I need to start from scratch that is to learn HDFS and move upwards, so anything that can help me when the situation is like this ?
- heartsucker 10y agoI read the O'Reilly books on Hadoop and Hvae in 2013. That was helpful, but I still think Hive is a good place to start. If you know how a join and an aggregate work, you can see how Hadoop turns those into into M/R operations. If you really have to learn how Hadoop works, you'll probably have to look at the source. Write the Hello World program and get it to run on AWS EMR (as others have suggested). Then clone the source to Hadoop and open it in your favorite editor. If you have to understand the nuts and bolts, the source code is the way to go. If you only have to know how to use it, then a book + online tutorial + simple project will get you up to speed within a week or two.
- hubatrix 10y agoGuess I'll do that, thanks !!