3 ms·
Relatedly, why are you advocating using MLLib/RDDs when they have been deprecated in favor of ML/DataFrames (http://spark.apache.org/docs/latest/ml-guide.html h
by nl 10y ago
Relatedly, why are you advocating using MLLib/RDDs when they have been deprecated in favor of ML/DataFrames (http://spark.apache.org/docs/latest/ml-guide.html http://spark.apache.org/docs/latest/ml-guide.html)?
Note that the DataFrame version of MLLib (which is still called MLLib btw, even if the package is different) isn't feature complete compared to the RDD version.
For example, the DataFrame version of LogisticRegression is binary only, while the RDD one allows multiple classes. I guess the suggested work around is to use the OneVsRest classifier (in Spark 2.0), but that blows the memory out on fairly small tasks.
I like Spark a lot, but MLLib isn't as strong as it appears at first glance.