3 ms·
I read the post with great interest but it turned out to be a provocative but shallow trolling that can be summarized as, "R is better because it is better LOl!
by srean 12y ago
I read the post with great interest but it turned out to be a provocative but shallow trolling that can be summarized as, "R is better because it is better LOl!" I was expecting something more illuminating.
It says or shows nothing new, does not addresses the well known complaints that users have against R: performance per resources used, ease of achieving correctness.
"...built by statisticians, for statisticians" can be a dangerous place to take comfort in, when there are well aired problems in scalability, running costs, maintenance costs.
If one computes stats on 600 data points with 10 dimensions and feels king of the hill, please continue, but there is likelihood that some one else will be eating your lunch and you will be left behind. Quite sadly, this has already happened and is quite evident if one steps out of the stats bubble. Statistics could have been what machine learning and datamining is now, been the main driving force, the owner of initiative. On the contrary other communities are using statistics and probability motivated approaches but engineering them well to grab (funding) attention, well deserved in my opinion. It is them who are pushing the frontier of influence.
I have not taken a wholesale plunge into Julia yet, but to me one of its significant aspects that I think does not get enough attention among its many nice features is the break away from the "vectorization" paradigm.
Complain about speed to a Numpy or Octave person and a common stock response would be "but it dispatches to high-speed precompiled C loops and so is just as fast." This ignores the fact that these precompiled loops are typically too general and target the worst possible data patterns and have to be defensive. Thus they have do more work than a corresponding loop on array a C, C++ or a Fortran programmer would have written. Further, the vectorization approach typically requires more loops and more memory to fill out, not only those temporaries that are created within the vectorization operations, but extra objects required to write the operation as a vector expression in the first place.
Numpy's broadcast does eliminate a lot of extra memory allocation at the cost of some more indexing work. Matlab did not have this for the longest time. Another Numpy tool that is great is numexpr http://code.google.com/p/numexpr/ http://code.google.com/p/numexpr/. So much so that if a colleague complains about numpy being slow I playfully refuse to take a look till whatever can be numexper'ed has been numexper'ed. It tries to elide temporaries and parallelize operations. The general bottomline remains, (i) individual primitives have to do more work and (ii) the primitives have to be called more often (iii) needs more memory copies.
Julia's approach is quite fresh in this regard, so is Haskell's loop fusion based array primitives. The problem with Haskell is that these optimizations can be very opaque unless you are an expert. A small change can make your code go 30 times faster, and similarly a small change can make it 30 times slower.
An aspect of vectorization that I do like a huge lot is their terse expressiveness. Code is a lot shorter and once you are used to it, such code is easy to read. So I thought I would miss this aspect in Julia, but then I was pointed to devectorize.jl on HN and its aimed to address this very issue.
- deleted 12y ago[deleted]