2 ms·
This is almost apples and oranges. Mahout's power lies in it's ability to handle huge amounts of data, in a parallel fashion. Weka (which is rarely used these d
by discardorama 12y ago
This is almost apples and oranges. Mahout's power lies in it's ability to handle huge amounts of data, in a parallel fashion. Weka (which is rarely used these days anyways) is for smaller problems and experimentation.
None of these (Mahout and Weka) are mainstream anymore. For large-scale classification, people are using packages like VW[1] . And for small-scale experimentation, SciKit or R.
[1] http://hunch.net/~vw http://hunch.net/~vw
- riffraff 12y agoIIRC VW doesn't have the easy integration with the hadoop ecosystem that mahout does, I am not sure everyone has moved to VW.
- jbooth 12y agoDoesn't need it. If you're computing a linear/logistic regression via gradient descent, performance-oriented C code on a single machine using local filesystem/caches will beat a hadoop-based algorithm for just about any size dataset.
- riffraff 12y agoI meant that shops are likely to already have data sitting in hdfs/hive/whatever which they can trivially use with mahout, while they may not have it sitting on a single file system on a beefy machine.
- discardorama 12y agoVW works on Hadoop just fine, though it's not very "clean" and native. But it works.
- jbooth 12y agoThey can hadoop fs -get it or query it from hive. Because you want to coalesce all your model updates for every pass over the data, the long startup-time for hadoop jobs actually plays into this. You can have hundreds of millions of samples, in a sparse space of hundreds of millions of parameters, and do it faster on a single node using VW than in a hadoop cluster of equivalent nodes running Mahout.