4 ms·

Out of curiosity, why are you so desperate to get away from R? Is it for the speed of a high performance language? As you mention, the vast selection of librar

by izyda 12y ago

Out of curiosity, why are you so desperate to get away from R? Is it for the speed of a high performance language? As you mention, the vast selection of libraries (and the continual development of these libraries) make even the python module universe look small (in reference to statistical methods, statistical algorithms/machine learning, plotting). Further, not only are there a lot of libraries.. but many of them are bleeding edge written by the same people who originally invented the given algorithm. Most similar libraries in other languages (like Python) have incomplete ports of these libraries to them often not written by the same people. Not to mention, the documentation behind these algorithms is often excellent - besides the usual man files, there's "vignettes" (detailed instruction manual) as well as an entire journal that now has become a place to publish papers detailing the usage and implementation of many of these algorithms. So I'm curious, why leave?

zmmmmm 12y ago
I love a lot about R, and I totally acknowledge the benefits. I don't think I would ever stop using it for data exploration and ad hoc analysis. But after trying really hard over the past 3 years I'm sort of giving up ever being able to use it in a productive reliable fashion for building more complex software. It's a failing on my own part in many ways, but the loose typing and poor support for structuring code interact very badly with my personal style. When I'm working in R I literally spend 80% of my time fighting with the type system, debugging weird and wonderful features of R. I'm at the point where whenever I write an R function the first 50 lines of the code are type checks to make sure the data coming in is what I expect. I came to this point after I started systematically tracking why I was wasting so much time, and nearly every time it would come back to the type of my data being something other than what I had expected or assumed. So I'm sort of figuring if I'm at the point where I'm writing manually statically typed code in R ... I should look around for a language that's just like R but has at least slightly stronger data types built in.
- epaladin 12y ago
  I agree completely on the type system. When first starting with it, I was thinking "wow this is great, I don't have to worry about types and things just work". But then weird things start to happen, and you realize it's due to strange type issues that are sometimes harder than they should be to track down. And then half your code ends up being something like as.numeric(as.character(as.vector(xyz)))
srean 12y ago
I cannot speak for zmmmmm but I can think of two reasons (i) performance, not that R cant be fast but you have to pull out heavy machinery such as parallelization much earlier and (ii) its inconsistent and confusing language semantics, and silent and unexpected type promotion/demotions that make it very error prone to develop and debug code in. To me the one redeeming quality about R is ggplot and, ok, availability of push button libraries I am not so sure about the scope of the 'bleeding edge' part. It is popular among old school, statisticians. Another population (no pun intended) that R is popular in is one where one knows neither statistics nor machine learning but just wants to try out a laundry list of canned methods without a need to understand: pretty plot goes up, yay awesome; pretty plot goes down, ok try the next algorithm (or the other way round). Here I think R is pretty unbeatable in its breadth. It is not that popular among machine learners. Part of it is cultural, a typical machine learning person will be coming from a CS background and then R grates more. It has been claimed that R is lisp meets stats, I think that would be Julia now and Lush then. http://lush.sourceforge.net/ http://lush.sourceforge.net/. To quote Yann LeCun: "Lush combines three languages in one: a very simple to use, loosely-typed interpreted language, a strongly-typed compiled language with the same syntax, and the C language, which can be freely mixed with the other languages within a single source file, and even within a single function."
- izyda 12y ago
  You definitely have some good points - I agree on points (i) and (ii) - these are shortfalls of the language and in many cases (especially when building production level code) may be reasons to switch). I disagree with you premise that R is not popular among machine learners and the implication that has for the "bleeding edge" comment. Perhaps it is less popular with ML people coming from a CS background - but it seems to be the most widespread language for ML (or "statistical learning") among statistics people. And that fact is not to be discounted - perhaps the preeminent book on machine learning (or at the very least one of the most popular) - "The Elements of Statistical Learning" - is written by statisticians (and in fact uses R exclusively!). The Journal of Statistical software, which features papers detailing many ML libraries, has far larger coverage of R packages than any other language: http://www.jstatsoft.org/ http://www.jstatsoft.org/ . I would suggest this is the evidence for the "bleeding edge" comment. Other languages' packages, say Python's ML libraries (scikit-learn, pyBrain, etc.), do note even come close to the breadth of capability in this space. Coming from a statistics background,- I would venture to ask.. what's wrong with "pushbutton" libraries? And why are they only useful to people who do not know/understand much about ML/statistics? Say you have a dataset and you believe ,say, a Random Forest would be well suited to predicting some response. Let's assume you have a very good understanding of Random Forests. Why would you not want a push-button library? Why would you WANT to recode the thing yourself? It's not as if your Random Forest will be any better or "more correct" than the one on CRAN or whatever language's repository you are using. I would argue it's more likely to have mistakes (coding something like this from scratch is no small task and the ones on CRAN have often gone through many iterations, improvements, and reviews from many experts in the field). And let's say you have some domain knowledge or some informed belief about how the Random Forest needs to be adapted to your particular problem (a different loss function or something) - you can easily edit the R package source code to do that - no need to rebuild the car just to give it a new paint job...

by izyda 12y ago

zmmmmm 12y ago

I love a lot about R, and I totally acknowledge the benefits. I don't think I would ever stop using it for data exploration and ad hoc analysis. But after trying really hard over the past 3 years I'm sort of giving up ever being able to use it in a productive reliable fashion for building more complex software. It's a failing on my own part in many ways, but the loose typing and poor support for structuring code interact very badly with my personal style. When I'm working in R I literally spend 80% of my time fighting with the type system, debugging weird and wonderful features of R. I'm at the point where whenever I write an R function the first 50 lines of the code are type checks to make sure the data coming in is what I expect. I came to this point after I started systematically tracking why I was wasting so much time, and nearly every time it would come back to the type of my data being something other than what I had expected or assumed. So I'm sort of figuring if I'm at the point where I'm writing manually statically typed code in R ... I should look around for a language that's just like R but has at least slightly stronger data types built in.

epaladin 12y ago
I agree completely on the type system. When first starting with it, I was thinking "wow this is great, I don't have to worry about types and things just work". But then weird things start to happen, and you realize it's due to strange type issues that are sometimes harder than they should be to track down. And then half your code ends up being something like as.numeric(as.character(as.vector(xyz)))

srean 12y ago

I cannot speak for zmmmmm but I can think of two reasons (i) performance, not that R cant be fast but you have to pull out heavy machinery such as parallelization much earlier and (ii) its inconsistent and confusing language semantics, and silent and unexpected type promotion/demotions that make it very error prone to develop and debug code in. To me the one redeeming quality about R is ggplot and, ok, availability of push button libraries I am not so sure about the scope of the 'bleeding edge' part. It is popular among old school, statisticians. Another population (no pun intended) that R is popular in is one where one knows neither statistics nor machine learning but just wants to try out a laundry list of canned methods without a need to understand: pretty plot goes up, yay awesome; pretty plot goes down, ok try the next algorithm (or the other way round). Here I think R is pretty unbeatable in its breadth. It is not that popular among machine learners. Part of it is cultural, a typical machine learning person will be coming from a CS background and then R grates more. It has been claimed that R is lisp meets stats, I think that would be Julia now and Lush then. http://lush.sourceforge.net/ http://lush.sourceforge.net/. To quote Yann LeCun: "Lush combines three languages in one: a very simple to use, loosely-typed interpreted language, a strongly-typed compiled language with the same syntax, and the C language, which can be freely mixed with the other languages within a single source file, and even within a single function."

izyda 12y ago
You definitely have some good points - I agree on points (i) and (ii) - these are shortfalls of the language and in many cases (especially when building production level code) may be reasons to switch). I disagree with you premise that R is not popular among machine learners and the implication that has for the "bleeding edge" comment. Perhaps it is less popular with ML people coming from a CS background - but it seems to be the most widespread language for ML (or "statistical learning") among statistics people. And that fact is not to be discounted - perhaps the preeminent book on machine learning (or at the very least one of the most popular) - "The Elements of Statistical Learning" - is written by statisticians (and in fact uses R exclusively!). The Journal of Statistical software, which features papers detailing many ML libraries, has far larger coverage of R packages than any other language: http://www.jstatsoft.org/ http://www.jstatsoft.org/ . I would suggest this is the evidence for the "bleeding edge" comment. Other languages' packages, say Python's ML libraries (scikit-learn, pyBrain, etc.), do note even come close to the breadth of capability in this space. Coming from a statistics background,- I would venture to ask.. what's wrong with "pushbutton" libraries? And why are they only useful to people who do not know/understand much about ML/statistics? Say you have a dataset and you believe ,say, a Random Forest would be well suited to predicting some response. Let's assume you have a very good understanding of Random Forests. Why would you not want a push-button library? Why would you WANT to recode the thing yourself? It's not as if your Random Forest will be any better or "more correct" than the one on CRAN or whatever language's repository you are using. I would argue it's more likely to have mistakes (coding something like this from scratch is no small task and the ones on CRAN have often gone through many iterations, improvements, and reviews from many experts in the field). And let's say you have some domain knowledge or some informed belief about how the Random Forest needs to be adapted to your particular problem (a different loss function or something) - you can easily edit the R package source code to do that - no need to rebuild the car just to give it a new paint job...