3 ms·
It will be interesting to see how all the R libraries play with Spark. There are bound to be some hiccups there.
by IndianAstronaut 11y ago
It will be interesting to see how all the R libraries play with Spark. There are bound to be some hiccups there.
- minimaxir 11y agoMy interpretation is that it will convert DataFrames to normal data.frames when necessary. Unfortunately, this removes the performance efficiency of Spark. Since currently SparkR only supports aggregation, it limits the usability of SparkR slightly. Future versions will apparently have MLib support which should alleviate that.