5 ms·

Why did R take over the world of statistics anyways? I remember about 8 years ago I had an interest in statistics. Everything was SAS, SPSS etc which i just did

by apalmer 12y ago

Why did R take over the world of statistics anyways? I remember about 8 years ago I had an interest in statistics. Everything was SAS, SPSS etc which i just didnt have the budget for... I happened to find R as it was a freely available for linux, impression I got was no one used it, well some limited use in academia, but no professional usage. Fast forward it seems like the most widely used tool in computer data analysis. Did something fundamentally change?

stillsut 12y ago
Packages. They can be built by a single prof/grad student. They can be used by anyone based on R's package-manager GUI. Research in most fields is going to move more quickly than any software team can add functionality to monolithic programs like SAS. Therefore, most functionality becomes available only in R. Users - people who want to run specialized-domain-function-X on their dataset are best suited by learning R, then using their colleagues packages.
- rcthompson 12y ago
  I don't know much about SAS or SPSS, but do they really not have a usable package/module system for code reuse? Or is it just not as good as R's?
  - claytonjy 12y ago
    I don't know about the package system for SAS or SPSS (though I am aware that papers get published with code for said platforms), but I think the bigger barrier is availability; R is free and open.
  - _deh 12y ago
    SAS a bit, but as another comment notes, nothing like CRAN. SPSS not at all, at best you could publish some sort of script.
- _deh 12y ago
  Agree entirely. Academics led and industry followed.
rcthompson 12y ago
I think you answered your own question: "...SAS, SPSS etc which i just didnt have the budget for... I happened to find R as it was a freely available..."
Mikeb85 12y ago
People's attitudes towards open source changed, no doubt thanks to vendors like IBM, Oracle and others, who embraced R (and open source as a whole). Also, R Studio has made the entry barrier to R very low...
- data_spy 12y ago
  Agreed on RStudio being a factor. For me its all about packages and RStudio. I totally believe some other languages are better in general (Python), but RStudio makes a difference at work. As I can quickly show co-workers charts, data views and Shiny Apps.
mahmud 12y ago
1) It's free. 2) CRAN, nothing like it for SAS & SPSS. 3) Early adoption by book authors. A whole generation of data scientists were taught in R.
- rcthompson 12y ago
  I remember back in college that Microsoft was pretty much giving away student editions of all their software. I think they realized that if students, who are invariably on a budget, couldn't get access to MS software, they'd learn to use the FOSS replacements instead, and by the time they graduated and entered the workforce they would know the free stuff better and start spreading it. I guess SAS and SPSS didn't have any similar kind of strategy.
  - blumkvist 12y ago
    You are guessing wrong.
    - rcthompson 12y ago
      Fair enough, I wasn't heavily into stats programming at the time, so I wouldn't necessarily have heard of such a program from SAS or SPSS.
- _deh 12y ago
  To expand on the free point - it's not just that SAS and SPSS have a price tag. You're then supposed to pay for a load of bolt-on modules. The extreme case is if you want to do decision trees in SAS, they've made a commercial decision that you must be doing some kind of 'Enterprise-Grade Data Mining', and therefore need to pay their 5-figure price tag rather than their 4-figure price tag. I think this has been a big barrier to any sort of package ecosystem to rival CRAN. Even if your entire target audience has SAS or SPSS, if your new package depended on one or more of those modules, it's going to cost some of your prospective users hard cash to run it. Whereas R just installs the dependencies for you.
grayclhn 12y ago
1) compatibility with S-Plus, which was widely used by statisticians for long before R 2) developing new statistical methods and other innovations (knitr, etc) is much easier in R or S-Plus than SAS, SPSS, etc. So R's improved a lot faster than the others.
rm999 12y ago
The biggest boon for R was the rise of "data science". Data scientists have actually existed for decades, but they had different titles (quants, actuaries, quantitative researchers, etc) and came from fields that preferred enterprise software for various reasons. Data scientists, on the other hand, largely rose from tech companies that hired software engineers and preferred open-source software. When I started in the field ~10 years ago there were very few reasonable open-source options outside of R. Matlab and SAS were prohibitively expensive, Octave was too immature, Python didn't have basic functionality like data frames. In short, it was our only viable option, so we used it. A lot of people started using it for the same reasons, so R's library support became the best. This is what keeps me semi-locked into R - I'll probably eventually move to Python or Julia, but R hit the critical mass first and in a huge way.
- grayclhn 12y ago
  R's library support predates your timeline.
  - rm999 12y ago
    The most impactful, important packages I use were all developed after 2005: ggplot2, plyr, glmnet, gbm, reshape, knitr. Out of curiosity, which ones are you referring to? edit: I may be using the word library imprecisely here, which may be causing miscommunication. I mean 'packages', to be clear.
    - Blahah 12y ago
      Prior to those packages, R was the language in which most new statistical methods would be published. So it has had very good library support for statistics for a long time. Only more recently has it also gained excellent data manipulation support.
      - disgruntledphd2 12y ago
        Which kinda violates the principles expressed by the writers of said open source project, but I'm all for it on the basis that if I never again have to use base reshape, stack or gsub then it would be a good thing. I believe An Introduction to R specifically notes that one should use other software to provide R with appropriate data.
x0x0 12y ago
R won because of 1 - very high usability, particularly (contrast to python) for users who aren't developers, don't think like developers, and don't want to be developers 2 - very high expressibility 3 - it's free, so you can install it on lab computers and your laptop and everywhere else without forking out tons of money (see the price of matlab) 4 - a generate of grad students used it and it, in turn, was taught to a generate of undergrads 5 - academic statisticians tend to like it and tend to implement stats techniques in it, so there's a wide body of shared code on cran implementing most analyses you can think of
- ak217 12y ago
  It's funny that you mention usability and expressibility (expressiveness?) as R's upside compared to Python. I've used both extensively, and in my experience, R is horrendously bad at both, and Python blows it out of the water.

by apalmer 12y ago

stillsut 12y ago

Packages. They can be built by a single prof/grad student. They can be used by anyone based on R's package-manager GUI. Research in most fields is going to move more quickly than any software team can add functionality to monolithic programs like SAS. Therefore, most functionality becomes available only in R. Users - people who want to run specialized-domain-function-X on their dataset are best suited by learning R, then using their colleagues packages.

rcthompson 12y ago
I don't know much about SAS or SPSS, but do they really not have a usable package/module system for code reuse? Or is it just not as good as R's?
- claytonjy 12y ago
  I don't know about the package system for SAS or SPSS (though I am aware that papers get published with code for said platforms), but I think the bigger barrier is availability; R is free and open.
- _deh 12y ago
  SAS a bit, but as another comment notes, nothing like CRAN. SPSS not at all, at best you could publish some sort of script.
_deh 12y ago
Agree entirely. Academics led and industry followed.

rcthompson 12y ago

I think you answered your own question: "...SAS, SPSS etc which i just didnt have the budget for... I happened to find R as it was a freely available..."

Mikeb85 12y ago

People's attitudes towards open source changed, no doubt thanks to vendors like IBM, Oracle and others, who embraced R (and open source as a whole). Also, R Studio has made the entry barrier to R very low...

data_spy 12y ago
Agreed on RStudio being a factor. For me its all about packages and RStudio. I totally believe some other languages are better in general (Python), but RStudio makes a difference at work. As I can quickly show co-workers charts, data views and Shiny Apps.

mahmud 12y ago

1) It's free. 2) CRAN, nothing like it for SAS & SPSS. 3) Early adoption by book authors. A whole generation of data scientists were taught in R.

rcthompson 12y ago
I remember back in college that Microsoft was pretty much giving away student editions of all their software. I think they realized that if students, who are invariably on a budget, couldn't get access to MS software, they'd learn to use the FOSS replacements instead, and by the time they graduated and entered the workforce they would know the free stuff better and start spreading it. I guess SAS and SPSS didn't have any similar kind of strategy.
- blumkvist 12y ago
  You are guessing wrong.
  - rcthompson 12y ago
    Fair enough, I wasn't heavily into stats programming at the time, so I wouldn't necessarily have heard of such a program from SAS or SPSS.
_deh 12y ago
To expand on the free point - it's not just that SAS and SPSS have a price tag. You're then supposed to pay for a load of bolt-on modules. The extreme case is if you want to do decision trees in SAS, they've made a commercial decision that you must be doing some kind of 'Enterprise-Grade Data Mining', and therefore need to pay their 5-figure price tag rather than their 4-figure price tag. I think this has been a big barrier to any sort of package ecosystem to rival CRAN. Even if your entire target audience has SAS or SPSS, if your new package depended on one or more of those modules, it's going to cost some of your prospective users hard cash to run it. Whereas R just installs the dependencies for you.

grayclhn 12y ago

1) compatibility with S-Plus, which was widely used by statisticians for long before R 2) developing new statistical methods and other innovations (knitr, etc) is much easier in R or S-Plus than SAS, SPSS, etc. So R's improved a lot faster than the others.

rm999 12y ago

The biggest boon for R was the rise of "data science". Data scientists have actually existed for decades, but they had different titles (quants, actuaries, quantitative researchers, etc) and came from fields that preferred enterprise software for various reasons. Data scientists, on the other hand, largely rose from tech companies that hired software engineers and preferred open-source software. When I started in the field ~10 years ago there were very few reasonable open-source options outside of R. Matlab and SAS were prohibitively expensive, Octave was too immature, Python didn't have basic functionality like data frames. In short, it was our only viable option, so we used it. A lot of people started using it for the same reasons, so R's library support became the best. This is what keeps me semi-locked into R - I'll probably eventually move to Python or Julia, but R hit the critical mass first and in a huge way.

grayclhn 12y ago
R's library support predates your timeline.
- rm999 12y ago
  The most impactful, important packages I use were all developed after 2005: ggplot2, plyr, glmnet, gbm, reshape, knitr. Out of curiosity, which ones are you referring to? edit: I may be using the word library imprecisely here, which may be causing miscommunication. I mean 'packages', to be clear.
  - Blahah 12y ago
    Prior to those packages, R was the language in which most new statistical methods would be published. So it has had very good library support for statistics for a long time. Only more recently has it also gained excellent data manipulation support.
    - disgruntledphd2 12y ago
      Which kinda violates the principles expressed by the writers of said open source project, but I'm all for it on the basis that if I never again have to use base reshape, stack or gsub then it would be a good thing. I believe An Introduction to R specifically notes that one should use other software to provide R with appropriate data.

x0x0 12y ago

R won because of 1 - very high usability, particularly (contrast to python) for users who aren't developers, don't think like developers, and don't want to be developers 2 - very high expressibility 3 - it's free, so you can install it on lab computers and your laptop and everywhere else without forking out tons of money (see the price of matlab) 4 - a generate of grad students used it and it, in turn, was taught to a generate of undergrads 5 - academic statisticians tend to like it and tend to implement stats techniques in it, so there's a wide body of shared code on cran implementing most analyses you can think of

ak217 12y ago
It's funny that you mention usability and expressibility (expressiveness?) as R's upside compared to Python. I've used both extensively, and in my experience, R is horrendously bad at both, and Python blows it out of the water.