3 ms·
Java's not actually a bad choice, considering some of the tools that are available. Hadoop[1], HBase[2], etc. Depending on what you are trying to accomplish of
by mindcrime 16y ago
Java's not actually a bad choice, considering some of the tools that are available. Hadoop[1], HBase[2], etc. Depending on what you are trying to accomplish of course, you could also include Lucene[3], Solr[4], Mahout[5] and Weka[6] in that list.
Of course cool Java libraries don't mean you have to work in Java. Scala[7] seems to be gaining quite a bit of momentum, as does Clojure[8].
From your reading list, it sounds like you may be interested in predictive analysis, machine learning, etc. If so, Python[9] is pretty popular in those circles and has some nice libraries available as well[10]. And you can do neat things like use a Hadoop cluster for map/reduce processing, even if you have a client written in Python (or whatever)[11].
Of course Map/Reduce[12] isn't the only distributed processing model out there. You might want to learn MPI[13] then explore the MPI bindings that are available for many (most?) languages, and look at other models of distributed / parallel computing.
Anyway, to summarize, I'd say that if you already know one or more of Java, Python, C, C++ or Scala, you're in good shape. If not, some combination of the above should fit the bill. And yes, R might be handy as well.
Not to take anything away from Erlang, Haskell, Forth, Factor, Scheme, Lisp, OCaml, Perl, or any other language of course. I'm just not as familiar with those so I can't say much about them.
1. http://hadoop.apache.org/core/ http://hadoop.apache.org/core/
2. http://hbase.apache.org/ http://hbase.apache.org/
3. http://lucene.apache.org/java/docs/ http://lucene.apache.org/java/docs/
4. http://lucene.apache.org/solr/ http://lucene.apache.org/solr/
5. http://mahout.apache.org/ http://mahout.apache.org/
6. http://www.cs.waikato.ac.nz/ml/weka/ http://www.cs.waikato.ac.nz/ml/weka/
7. http://www.scala-lang.org/ http://www.scala-lang.org/
8. http://clojure.org/ http://clojure.org/
9. http://www.python.org/ http://www.python.org/
10. http://www.google.com/search?q=python+machine+learning http://www.google.com/search?q=python+machine+learning
11. http://www.michael-noll.com/wiki/Writing_An_Hadoop_MapReduce_Program_In_Python http://www.michael-noll.com/wiki/Writing_An_Hadoop_MapReduce...
12. http://en.wikipedia.org/wiki/MapReduce http://en.wikipedia.org/wiki/MapReduce
13. http://en.wikipedia.org/wiki/Message_Passing_Interface http://en.wikipedia.org/wiki/Message_Passing_Interface
- evgen 16y agoThis is a good overview, but I would definitely plump Clojure higher in the implicit rankings of your list. If you know Java and Lisp does not frighten you then it is probably a better choice for big data analysis than Scala. The Incanter libraries give you a very nice R-like environment for numerical analysis and Cascalog is a pretty nifty system for interacting with Hadoop.
- mindcrime 16y agoYou're probably quite right. I didn't give Clojure a stronger endorsement only because I don't know it very well yet myself... I'm just starting with it. I wasn't aware of either Incanter or Cascalog, for example, until you mentioned them.