4 ms·
These days a cheap way to get attention seems to be to Hadoop'ize a well known but expensive operation. "Bogo-sort too slow ? no worries, we will run it on the
by srean 13y ago
These days a cheap way to get attention seems to be to Hadoop'ize a well known but expensive operation. "Bogo-sort too slow ? no worries, we will run it on the cloud." These approaches are buzzword compatible. They keep the marketing guys happy. Managers do not get fired for embracing the cloud/Hadoop, even if it is done in a grossly inefficient way.
I realize I bring unwelcome rain, but the post really begs the question if using R here is the best use of resources: technological and human
- ambrood 13y agoHey srean, This is Amey from Rescale here. I definitely agree with your larger point about R not being the most efficient tool to use in this case. But if you read the post closely, that isn't the point we are trying to make. R its other array-based cousins provide a much more natural and expressive framework for linear algebra/matrix operations. I'd say it is still worthwhile to try and scale your existing R code to moderately large datasets. Come to think of it, most "big data" isn't really that big.