11 ms·

Allow me space for an editorial and downvote magnets :P I'm actually a little concerned by the "phenomenon" of R. I say this as someone whose workplace uses b

Allow me space for an editorial and downvote magnets :P I'm actually a little concerned by the "phenomenon" of R. I say this as someone whose workplace uses both SAS and R. I'm also familiar with python, and wish we could use it more at work, but it doesn't have the "cult-following/network effect" amongst statisticians that R has. The "problem" I speak of is that R is very popular for people applying a quick little stats script for a package they've downloaded using a technique they don't understand with output they haven't verified on a tiny problem that won't scale. And 95%+ of users are just doing it by rote, and now they're trying to apply it to problems outside of its domain. But ACow_Adonis you say, doesn't that just describe everyone with every programming language ever? Yes. But you see, R seems almost designed (or not designed) as a language of unseen problems. It is several multiples slower than regular python (if you thought that possible), and several HUNDRED times worse than other compiled languages. It has no un-boxed primitive numbers. Let me just say that again. A language for numbers that doesn't have primitive unboxed numbers. It is the poster boy of Wirth's law. But not only that, i said its basically been designed for "dodgy results". Watch how its attempt at lexical scope combines with lazy evaluation for ridiculous fun. Bizarre, automatic and random conversions behind the scenes. 1-indexing of arrays...but 0-indexing doesn't throw an error. Automatic repetition of values in smaller arrays when combined with arrays of larger size. Internal functions of one letter names in a language with KIND OF one name-space for people dealing with MATH with a long history of using these individual letters for other things! So combine these "features" of the language with people implementing things by rote, not checking their results, returning results without error messages/warnings... A SAS marketing person made the comment once of “We have customers who build engines for aircraft. I am happy they are not using freeware when I get on a jet.” and we all piled-on the hate, and rightly so. But after using R, what scares me more is the thought that professional stats people ARE using it when i get on a jet :(

marcosscriven 13y ago
This absolutely resonates with me. As a side project at a client of mine, they wanted me to expose some R reports via a web interface. The reports themselves were incredibly slow - and did the most horrendous SQL queries. The reports could have been achieved in several other languages, and in much more performant ways, but it absolutely had to be R because that's all the analyst in question knew. From what I've seen of Julia it could be a massive contender for this kind of usage.
timr 13y ago
Amen. I've been using R since ~2000, and it's a terrible hack of a language. The syntax is quirky, the speed is underwhelming, and the memory usage makes it unsuitable for all but the smallest data analysis problems. It's a programming language designed by people who don't really know how to program. ...but if you know it, it lets you tap into this huge collection of statistical libraries with relatively little effort. So you're right: most people are using it as a glorified graphing calculator. But given that the alternative is (usually) implementing complicated, error-prone algorithms in another language, I'm still glad it exists.
blueblob 13y ago
I am using R for a fairly large project (because I am working with other people who are using R) and I really agree with this. R is easy to implement small things, but gluing them all together is awful. Additionally, the debugging features are not very good, especially if you created a package. Everything using Rcpp basically requires you to make a package and believe it or not the C/C++ is easier to debug than the R. I do believe that going the other way (Rinside) or using R in C++ is a much better solution in terms of efficiency, and getting the actual results you expect. I am, however, impressed with the amount and, in general, quality of R packages.
- scottlocklin 13y ago
  I used to think R's debugging was awful. Then someone told me about options(error="recover"), and I get a nice Lisp/Matlab like stack when something bad happens. Doesn't help with the C/C++ FFI piece, but that's not usually what is needed. There's a lot of little doodads like that in R. The real problem is there is no good book teaching people who need to write complex things how to do it properly.
  - blueblob 13y ago
    You get a stack trace but only if your code is not in a package.
yrochat 13y ago
It is sad that what you say has nothing to do with the article linked, which has interesting content for science as a whole---not just R. Instead of being positive and considering the community, you just bring a negative and unrelated message (that silly "airplane anecdote"). So yes, sad.
epaladin 13y ago
I use R almost every day. I'm tempted to start porting Bioconductor packages to Julia, so I can go back to working in languages that don't make my brain melt. It's scary when you realize you actually got used to working around the hacks/quirks of R.
- ihnorton 13y ago
  FYI in case you haven't seen it: https://github.com/BioJulia/Bio.jl https://github.com/BioJulia/Bio.jl (nascent bio community with some good people onboard already; there are a handful of other bio-related things in the Julia pkg repository that have not migrated under the BioJulia umbrella yet)
Homunculiheaded 13y ago
The thing with R that I think is important to note is that you don't have interactivity to support code (eg in ruby and python the huge advantage of the repl is to interact with the code you are writing while you're writing it) but rather your code is a way to the make interactive experience better. R is an amazingly advanced calculator. The reason I use R over python most of the time is because, despite some amazing improvements in this area by the python community, there's no better tool for fluidly interacting with data that offers the same power. That said, R is not for writing software systems. People used to refer to many interpreted languages as "scripting languages", and while this is clearly not the case for Python and Ruby, this is exactly what R is. There's a good reason in RStudio it says "New File > R Script". The limit of using R is when you have a bunch of scripts that interact with each other to create a bunch of visualizations/reports. If your system gets more complicated then that, write it in something else. Of the many language/environment combos I've used, I don't think I've found one better for rapid prototyping than R, and following from that R has no place near anything that would be called "production". I also happen to think, if used properly, this is a good thing since it means your "prototype" never accidentally creeps into suddenly being your production system.
- civilian 13y ago
  Have there been any thoughts of making a python package that emulate's R's "ease of use and immediately having X statistical methods available"? I imagine it would mostly be a renaming wrapper around NumPy and SciPy.
  - misiti3780 13y ago
    try ipython w/ pandas and yhats ggplot plotting tool. as ggplot becomes fully functional, this combo will replace R for visualizations IMO
    - bentcorner 13y ago
      Link for the ggplot python port: https://github.com/yhat/ggplot/ https://github.com/yhat/ggplot/
epistasis 13y ago
> It has no un-boxed primitive numbers. Let me just say that again. A language for numbers that doesn't have primitive unboxed numbers. What you're leaving out of this makes me think that you don't understand the field at all. Everything is a vector. There's no need for an 'unboxed' number when you have vectors. If you're doing computation thinking of operating on individual data points rather than vectors, matrices, and multi-dimensional arrays of datapoints, you're doing it wrong. R is doing it right.
- zwieback 13y ago
  I think you may have unwittingly proved ACow_Adonis' point just now.
  - epistasis 13y ago
    If so, I have absolutely no clue what the point was. He referred to Wirth's Law, but the way to make software faster as the hardware gets faster is to abandon the idea of individual unboxed numbers, and move to vectors, as that's what the hardware is using. R, like APL, is much better in its intended domain for being a vector-based language rather than a scalar based language. If the application domain is a scalar based domain, then R is probably the wrong tool for the job. If somebody doesn't understand the cases where vector-based language is better, they've probably never encountered the right application domain for R.
    - zwieback 13y ago
      I think the original comment about unboxed numbers was not referring to vector vs. scalar but to boxing of primitive types (integers/floats) in object or other type wrappers: http://en.wikipedia.org/wiki/Object_type_(object-oriented_programming) http://en.wikipedia.org/wiki/Object_type_(object-oriented_pr.... It's easy to imagine situations where languages that directly support vector types could have performance benefits, especially on HW that supports vector instruction sets. On the other hand, languages that don't directly manipulate numbers in the native format of the CPU(s) will peform poorly for large datasets.
AngrySkillzz 13y ago
It's another one of those "worse is better" situations, I guess.
pseut 13y ago
I more or less agree with your specific points, but in the larger scheme of things I'm more concerned that these hypothetical people don't understand the stats than that the tool won't scale or is badly implemented. That said, it sure is a bitch of a language to try to develop for (since the consensus seems to be: write everything in C) and cran is a ghetto.
jjoonathan 13y ago
> Watch how its attempt at lexical scope combines with lazy evaluation for ridiculous fun. qplot(hp, mpg, data=subset(mtcars, mpg<30), shape=as.factor(cyl), color=am, facets=gear~cyl) vs mtbadcars <- subset(mtcars, mtcars$mpg<30) qplot(mtbadcars$hp, mtbadcars$mpg, shape=as.factor(mtbadcars$cyl), color= mtbadcars$am, facets=(mtbadcars$gear)~(mtbadcars$cyl)) Why thanks, I do find it ridiculously fun to use short, readable names while maintaining encapsulation so that I can 1) keep multiple versions of a dataset around 2) not prefix every reference to a column in the dataset with the dataset name, cluttering the argument list and requiring 20 replacements or a kludgy temporary variable every time I want to plot a different subset. Oh wait, you were trying to convince me that was a bad thing?
- pseut 13y ago
  Did you write qplot? (rhetorical question) A lot of these "quirks" of R are nice for end users when they're implemented well, but are unintuitive to program and, as a consequence, are inconsistently implemented across packages. Hadley doesn't write everything, you know.
  - jjoonathan 13y ago
    > Did you write qplot? (rhetorical question) Not sure what you're getting at. > A lot of these "quirks" of R are nice for end users when they're implemented well, but are unintuitive to program It's not a quirk, it's a core feature. It's used consistently and to great effect. Notice how the "subset" function takes advantage of the same flexibility. I'm about 95% sure neither the subset function nor the other standard library functions that use this "trick" were written by Hadley. The "trick" was expected to make expressions significantly easier to read and easier to write from the very beginning. It might surprise a few people who come from another language and think they've seen it all, but once they figure out what's going on (which should happen on the first tutorial or 2nd or 3rd copypaste) it'll be a pleasant surprise. Unless they kneejerk and hate on it because it's unusual among languages.
    - 13y ago
ACow_Adonis 13y ago
Also, allow me just a quick addon to which people probably aren't responding in the comments below. I assure you banks and the like using R aren't passing round all their R/python/C code either. R didn't bring open source to science. It brought free (as in price) software to stats, and that, along with its script-like ability to apply formulas quickly, its vast library, and its universal teaching in stats courses in universities, is the reason for its popularity. I'd even go so far as to say that python has had more of an influence outside of stats/bio/pharma. But spend a bit of time in the SAS community to which it is commonly contrasted and you'll see massive amounts of code sharing, examples and how-tos. The interesting thing is to observe how these things play together. I argue that sharing of source has less value if the run-time on which it operates is not available to you. Of course, SAS is so widespread in big business that you might point out that it quite clearly is available to a lot of people, and that it quite clearly is valuable to them, its just not available if you can't pay or aren't in a connected uni/job. I know my SAS and I can do several things in it that whip R and python's butt if the task is that which SAS is good for. It has its own separate and relevant issues in terms of design abd implementation. I can rail against all the tools i use :P But the high-entry-cost of the software itself is the prime reason I'm trying to turn my back on it (because I don't plan on being employed by a big company or being locked into a software vendor forever, and subsequently it is not available for me to use for my own projects, which are often more valuable/complex than the ones i'm writing for employment...). I imagine there are a large number of other programmers/hackers feeling the same way, and you might even say that its evidenced by the parallel (as in along-side SAS, not multithreading) success of R. Perhaps there is a symbiotic relationship, subsequently, between free as in price vs open source code. Who knows. I need a "free as in beer"....
- mathattack 13y ago
  R didn't bring open source to science. It brought free (as in price) software to stats This is a very powerful statement. Never underestimate the power of free. It is very hard to compete with. Look at the browsers.
LordHumungous 13y ago
Well yes, a dynamic interpreted scripting language will have all the drawbacks of a dynamic interpreted scripting language. If you are trying to build a large scale application that requires a lot of debugging using R, you have chosen the wrong tool for the job. R is not Python or C++, it's intended for writing data analysis and visualization scripts.
GFK_of_xmaspast 13y ago
R has its share of issues, but jesus christ SAS is a horror.
- mathattack 13y ago
  Any comments on what the issues with SAS were? Performance? Interface?
ForHackernews 13y ago
R's origins are in academia and its primary users are statistics professors doing research. They don't care (and don't need to care) about performance or scaling. If business users want a free, plug-and-play statistics package they can throw into their analytics stack, R is not the right tool for that job.

marcosscriven 13y ago

This absolutely resonates with me. As a side project at a client of mine, they wanted me to expose some R reports via a web interface. The reports themselves were incredibly slow - and did the most horrendous SQL queries. The reports could have been achieved in several other languages, and in much more performant ways, but it absolutely had to be R because that's all the analyst in question knew. From what I've seen of Julia it could be a massive contender for this kind of usage.

timr 13y ago

Amen. I've been using R since ~2000, and it's a terrible hack of a language. The syntax is quirky, the speed is underwhelming, and the memory usage makes it unsuitable for all but the smallest data analysis problems. It's a programming language designed by people who don't really know how to program. ...but if you know it, it lets you tap into this huge collection of statistical libraries with relatively little effort. So you're right: most people are using it as a glorified graphing calculator. But given that the alternative is (usually) implementing complicated, error-prone algorithms in another language, I'm still glad it exists.

blueblob 13y ago

I am using R for a fairly large project (because I am working with other people who are using R) and I really agree with this. R is easy to implement small things, but gluing them all together is awful. Additionally, the debugging features are not very good, especially if you created a package. Everything using Rcpp basically requires you to make a package and believe it or not the C/C++ is easier to debug than the R. I do believe that going the other way (Rinside) or using R in C++ is a much better solution in terms of efficiency, and getting the actual results you expect. I am, however, impressed with the amount and, in general, quality of R packages.

scottlocklin 13y ago
I used to think R's debugging was awful. Then someone told me about options(error="recover"), and I get a nice Lisp/Matlab like stack when something bad happens. Doesn't help with the C/C++ FFI piece, but that's not usually what is needed. There's a lot of little doodads like that in R. The real problem is there is no good book teaching people who need to write complex things how to do it properly.
- blueblob 13y ago
  You get a stack trace but only if your code is not in a package.

yrochat 13y ago

It is sad that what you say has nothing to do with the article linked, which has interesting content for science as a whole---not just R. Instead of being positive and considering the community, you just bring a negative and unrelated message (that silly "airplane anecdote"). So yes, sad.

epaladin 13y ago

I use R almost every day. I'm tempted to start porting Bioconductor packages to Julia, so I can go back to working in languages that don't make my brain melt. It's scary when you realize you actually got used to working around the hacks/quirks of R.

ihnorton 13y ago
FYI in case you haven't seen it: https://github.com/BioJulia/Bio.jl https://github.com/BioJulia/Bio.jl (nascent bio community with some good people onboard already; there are a handful of other bio-related things in the Julia pkg repository that have not migrated under the BioJulia umbrella yet)

Homunculiheaded 13y ago

The thing with R that I think is important to note is that you don't have interactivity to support code (eg in ruby and python the huge advantage of the repl is to interact with the code you are writing while you're writing it) but rather your code is a way to the make interactive experience better. R is an amazingly advanced calculator. The reason I use R over python most of the time is because, despite some amazing improvements in this area by the python community, there's no better tool for fluidly interacting with data that offers the same power. That said, R is not for writing software systems. People used to refer to many interpreted languages as "scripting languages", and while this is clearly not the case for Python and Ruby, this is exactly what R is. There's a good reason in RStudio it says "New File > R Script". The limit of using R is when you have a bunch of scripts that interact with each other to create a bunch of visualizations/reports. If your system gets more complicated then that, write it in something else. Of the many language/environment combos I've used, I don't think I've found one better for rapid prototyping than R, and following from that R has no place near anything that would be called "production". I also happen to think, if used properly, this is a good thing since it means your "prototype" never accidentally creeps into suddenly being your production system.

civilian 13y ago
Have there been any thoughts of making a python package that emulate's R's "ease of use and immediately having X statistical methods available"? I imagine it would mostly be a renaming wrapper around NumPy and SciPy.
- misiti3780 13y ago
  try ipython w/ pandas and yhats ggplot plotting tool. as ggplot becomes fully functional, this combo will replace R for visualizations IMO
  - bentcorner 13y ago
    Link for the ggplot python port: https://github.com/yhat/ggplot/ https://github.com/yhat/ggplot/

epistasis 13y ago

> It has no un-boxed primitive numbers. Let me just say that again. A language for numbers that doesn't have primitive unboxed numbers. What you're leaving out of this makes me think that you don't understand the field at all. Everything is a vector. There's no need for an 'unboxed' number when you have vectors. If you're doing computation thinking of operating on individual data points rather than vectors, matrices, and multi-dimensional arrays of datapoints, you're doing it wrong. R is doing it right.

zwieback 13y ago
I think you may have unwittingly proved ACow_Adonis' point just now.
- epistasis 13y ago
  If so, I have absolutely no clue what the point was. He referred to Wirth's Law, but the way to make software faster as the hardware gets faster is to abandon the idea of individual unboxed numbers, and move to vectors, as that's what the hardware is using. R, like APL, is much better in its intended domain for being a vector-based language rather than a scalar based language. If the application domain is a scalar based domain, then R is probably the wrong tool for the job. If somebody doesn't understand the cases where vector-based language is better, they've probably never encountered the right application domain for R.
  - zwieback 13y ago
    I think the original comment about unboxed numbers was not referring to vector vs. scalar but to boxing of primitive types (integers/floats) in object or other type wrappers: http://en.wikipedia.org/wiki/Object_type_(object-oriented_programming) http://en.wikipedia.org/wiki/Object_type_(object-oriented_pr.... It's easy to imagine situations where languages that directly support vector types could have performance benefits, especially on HW that supports vector instruction sets. On the other hand, languages that don't directly manipulate numbers in the native format of the CPU(s) will peform poorly for large datasets.

AngrySkillzz 13y ago

It's another one of those "worse is better" situations, I guess.

pseut 13y ago

I more or less agree with your specific points, but in the larger scheme of things I'm more concerned that these hypothetical people don't understand the stats than that the tool won't scale or is badly implemented. That said, it sure is a bitch of a language to try to develop for (since the consensus seems to be: write everything in C) and cran is a ghetto.

jjoonathan 13y ago

> Watch how its attempt at lexical scope combines with lazy evaluation for ridiculous fun. qplot(hp, mpg, data=subset(mtcars, mpg<30), shape=as.factor(cyl), color=am, facets=gear~cyl) vs mtbadcars <- subset(mtcars, mtcars$mpg<30) qplot(mtbadcars$hp, mtbadcars$mpg, shape=as.factor(mtbadcars$cyl), color= mtbadcars$am, facets=(mtbadcars$gear)~(mtbadcars$cyl)) Why thanks, I do find it ridiculously fun to use short, readable names while maintaining encapsulation so that I can 1) keep multiple versions of a dataset around 2) not prefix every reference to a column in the dataset with the dataset name, cluttering the argument list and requiring 20 replacements or a kludgy temporary variable every time I want to plot a different subset. Oh wait, you were trying to convince me that was a bad thing?

pseut 13y ago
Did you write qplot? (rhetorical question) A lot of these "quirks" of R are nice for end users when they're implemented well, but are unintuitive to program and, as a consequence, are inconsistently implemented across packages. Hadley doesn't write everything, you know.
- jjoonathan 13y ago
  > Did you write qplot? (rhetorical question) Not sure what you're getting at. > A lot of these "quirks" of R are nice for end users when they're implemented well, but are unintuitive to program It's not a quirk, it's a core feature. It's used consistently and to great effect. Notice how the "subset" function takes advantage of the same flexibility. I'm about 95% sure neither the subset function nor the other standard library functions that use this "trick" were written by Hadley. The "trick" was expected to make expressions significantly easier to read and easier to write from the very beginning. It might surprise a few people who come from another language and think they've seen it all, but once they figure out what's going on (which should happen on the first tutorial or 2nd or 3rd copypaste) it'll be a pleasant surprise. Unless they kneejerk and hate on it because it's unusual among languages.
  - 13y ago

ACow_Adonis 13y ago

Also, allow me just a quick addon to which people probably aren't responding in the comments below. I assure you banks and the like using R aren't passing round all their R/python/C code either. R didn't bring open source to science. It brought free (as in price) software to stats, and that, along with its script-like ability to apply formulas quickly, its vast library, and its universal teaching in stats courses in universities, is the reason for its popularity. I'd even go so far as to say that python has had more of an influence outside of stats/bio/pharma. But spend a bit of time in the SAS community to which it is commonly contrasted and you'll see massive amounts of code sharing, examples and how-tos. The interesting thing is to observe how these things play together. I argue that sharing of source has less value if the run-time on which it operates is not available to you. Of course, SAS is so widespread in big business that you might point out that it quite clearly is available to a lot of people, and that it quite clearly is valuable to them, its just not available if you can't pay or aren't in a connected uni/job. I know my SAS and I can do several things in it that whip R and python's butt if the task is that which SAS is good for. It has its own separate and relevant issues in terms of design abd implementation. I can rail against all the tools i use :P But the high-entry-cost of the software itself is the prime reason I'm trying to turn my back on it (because I don't plan on being employed by a big company or being locked into a software vendor forever, and subsequently it is not available for me to use for my own projects, which are often more valuable/complex than the ones i'm writing for employment...). I imagine there are a large number of other programmers/hackers feeling the same way, and you might even say that its evidenced by the parallel (as in along-side SAS, not multithreading) success of R. Perhaps there is a symbiotic relationship, subsequently, between free as in price vs open source code. Who knows. I need a "free as in beer"....

mathattack 13y ago
R didn't bring open source to science. It brought free (as in price) software to stats This is a very powerful statement. Never underestimate the power of free. It is very hard to compete with. Look at the browsers.

LordHumungous 13y ago

Well yes, a dynamic interpreted scripting language will have all the drawbacks of a dynamic interpreted scripting language. If you are trying to build a large scale application that requires a lot of debugging using R, you have chosen the wrong tool for the job. R is not Python or C++, it's intended for writing data analysis and visualization scripts.

GFK_of_xmaspast 13y ago

R has its share of issues, but jesus christ SAS is a horror.

mathattack 13y ago
Any comments on what the issues with SAS were? Performance? Interface?

ForHackernews 13y ago

R's origins are in academia and its primary users are statistics professors doing research. They don't care (and don't need to care) about performance or scaling. If business users want a free, plug-and-play statistics package they can throw into their analytics stack, R is not the right tool for that job.