10 ms·

How popular is R in general? I started learning it because I want to make an attempt to do some projects on Kaggle. Most people use Pandas, Seaborn, etc, whic

How popular is R in general? I started learning it because I want to make an attempt to do some projects on Kaggle. Most people use Pandas, Seaborn, etc, which I will also use. However, to me R appears like a little better Swiss Army Knife to do initial analysis. ggplot2, tidyverse, ... Any help leveling up would be appreciated.

kickout 6y ago
Probably the most popular non-software engineer language for working with data. Millions and millions of users that have no idea what this blog post is technically about (but is interesting nonetheless)
- NegativeLatency 6y ago
  Second after Excel?
  - deleted 6y ago
    [deleted]
  - kickout 6y ago
    Fair, although I would say Excel and R are serving two separate purposes. But yes, Excel is of course #1
    - wtetzner 6y ago
      > Fair, although I would say Excel and R are serving two separate purposes. It's true that they target different use cases overall (obviously with some overlap), but Excel tends to be used for lots of things that would be better handled with a different tool, because it's what people know.
- ISL 6y ago
  Matlab has a substantial userbase, too.
  - kickout 6y ago
    Agreed, but IMO R >> MatLab and SAS over the past ~10 years. Both Matlab (physics??) and SAS (pharma/financial) seem to have further sunk into deep niches.
    - mjn 6y ago
      Matlab is still big in engineering as well. Matlab+Simulink in particular seems to have a fairly entrenched niche.
- ImaCake 6y ago
  I think this nails it. Given both python and R, these people will pick R because the motivation for using R is very clear, it does data analysis and statistics. Whereas python kinda does everything and that makes it a bit more tricky to understand.
J253 6y ago
In my experience I’ve seen R used in more exploratory/ad hoc type analysis and algorithm development by “non-developers”—-statisticians, scientists, etc. usually without performance consideration—-and that code is then turned into production code with the dev team using Python or C or something more performant or maintainable.
- pyromine 6y ago
  R is a nasty nasty language for productionalizing things, honestly it's just too flexible and let's you do the craziest things. But being so flexible makes it really expressive for doing ad-hoc analysis where you really don't know what you're looking for yet.
  - laichzeit0 6y ago
    It’s not just the language/runtime though. It’s the entire ecosystem around that that’s required to productionize it in any modern sense of the word. It’s hard to get right even in Python. I mean Flask (even with restx) still doesn’t even generate Swagger 3 documentation.
jhfdbkofdcho 6y ago
Extremely popular and widely used. Pandas etc are Python implementations of R constructs
ekianjo 6y ago
Very popular in academia, moderately popular in industry when it comes to data science/analysis. In any case, very powerful while Python has certainly numerous advantages over it.
vmchale 6y ago
Lots of scientists like it, psychologists and statisticians and such.
FranzFerdiNaN 6y ago
I work with people who mostly have a background in the social sciences or humanities and who work in R pretty much every day. They dont see themselves as programmers and Python is complete gibberish for them, while R just makes sense. When i meet people from other companies in roughly the same space (i work in healthcare doing data analysis), it's mostly the same. I actually meet more people who use SAS/SPSS than Python. For data analysis, R is in my opinion better than Python. It's when you have to integrate it in existing workflows that Python quickly becomes a better choice.
- lottin 6y ago
  It's not so much that Python is gibberish but that is written, as far as I know, predominantly by engineers, who aren't really experts in statistics, or experts in science for that matter. A scientist will tend to trust more code written by another scientist than code written by an engineer. At least, I would.
  - milesvp 6y ago
    I find this statement interesting. Historically scientists have a reputation for writing relatively poor code. Code that runs really slowly due to things like unintended nested loops, or striding values (x,y vs y,x). And code that doesn't handle non-happy path cases very well. Are you saying that you trust the code more because the domain knowledge make it more likely to get the right answer then? Has general knowledge increased such that scientists' code isn't as painful as it was 20 years ago?
    - lottin 6y ago
      Yeah, exactly, even though it might be terribly inefficient, I still trust scientific code more when it's written by scientists than when it's written by non-scientists, in terms of getting the right answer.
- bigger_cheese 6y ago
  Similar story for me. I am an Engineer (the non software type). I work at an industrial plant. We use SAS pretty extensively for data analysis, time series analysis, multivariate regressions etc. As well as for BI type stuff (reports, graphing, adhoc queries). For a while R was being pushed pretty heavily as a SAS alternative. My org paid R training courses etc. I found R and SAS pretty comparable at least the R packages we looked at (dpylr, ggplot2 etc). I know about Python, the programming language I used PyGTK back in the day to build GUI apps. But it would not be my first thought for doing data analysis work. Does Python even offer something like R studio/ SAS Enterprise Guide and does it have a trending package?
minimaxir 6y ago
As a data scientist who is proficient in both Python and R ecosystems, in my opinion R/tidyverse is substantially better for ad hoc EDA and data visualization. However, Python is better for nearly everything else in the field (namely, working with nontabular data, external APIs, deep learning, and productionization). It's about knowing which tool to use.
- omarhaneef 6y ago
  Ditto (not that I am proficient, but my experience matches). However, because the rest is easier in python, and my mental gears grind when I switch from one to the other, I end up using Python for the adhoc EDA and viz, and with Spyder, it is a pretty decent experience.
- disgruntledphd2 6y ago
  > (namely, working with nontabular data, external APIs, deep learning, and productionization). I agree with all of that except for productionisation. I would have agreed before dealing with issues around getting consistent versions of python + libraries to run. The issues I see with Python are as follows: - pip doesn't actually check to make sure your dependencies are compatible, which causes real problems with numpy et al - conda isn't available by default, and running it on remote boxes is non-trivial (I spent a whole week figuring out how to get it running in a remote non-login shell). - This makes it really, really difficult to actually get a standard set of libraries to depend upon, which is really important for production. R, on the other hand, actually resolves dependencies in its package manager, and the R CMD BUILD for packages, while super annoying helps you produce (more) portable code (did you know that conda doesn't provide cross-platform yml files unless invoked specifically?). In terms of handing it over to engineering/non data science people though, Python is much much much better. tl;dr Python's an ace language with a terrible production story.
  - ekianjo 6y ago
    > R, on the other hand, actually resolves dependencies in its package manager And you have packages like renv which also help isolate specific versions of packages to make portable environments even more reliable.
ryanar 6y ago
I love using R for exploratory work. Hadley Wickham's TidyVerse of packages make everything so ergonomic to use.
currymj 6y ago
think of it like shell scripting for statistics, although not nearly as limited as bash is compared to other programming languages. it works best if it's used semi-interactively, as a glue language between statistical packages which may be written in other languages. or to write simple "batch" scripts that basically just run a bunch of procedures in a row. RStudio makes the whole experience much nicer in terms of plotting, and RMarkdown is great for preparing documents. of course like shell scripting you can write fairly complicated programs in it, and sometimes people do, but due to backwards compatibility and weird design choices meant to make interactive use easier, programming "in the large" can get weird. the analogy works for Python too -- it is definitely reasonable to use Python for shell scripting, but using Python interactively to pipe things from one program to another is slightly more frustrating than doing it in the shell, although might be preferred due to its other advantages.
totalperspectiv 6y ago
More popular than I wish it was. It is the bash of the data science world. Totally ubiquitous and kind of a dumpster fire.
- dcolkitt 6y ago
  I disagree, the language is extremely powerful for interactive data exploration. A terse one-liner is all it takes to compute something like "what's the correlation between number of children and home size for people over 45 who live in counties with income variance at the 90th percentile weighted by population". Not that pandas/scipy/numpy don't make an admirable job. You can do something like this, but it's nowhere near as ergonomic as it is R. At the end of the day, R is fundamentally a language for data exploration, whereas with python those facilities are bolted on top of a general purpose environment.
- tharne 6y ago
  This is the best description of R that I've ever come across, and I say that as someone who learned R as their first programming language. Big mistake, btw. It took me years to unlearn all of the terrible habits I picked up from the R world. Do yourself a favor and start with python, if only to learn proper programming practices and techniques before diving into R.
  - uomopertica 6y ago
    While I agree that Python is better than R for programming etiquette, I would argue that proper programming practices and techniques are better learned in languages with static typing and proper variable scoping. Do yourself another favor and also look into C#, Swift or even Java.
    - coward8675309 6y ago
      If R is a reasonable tool for a given problem, C# or Swift or Java almost certainly will not be. The realistic alternatives to R are other numerical analysis packages, Julia, and Python. “The” answer for any given person or project is likely to be a function of your colleagues and peer group, your problem domain, your library needs. One of course is allowed to learn more than one thing. Maybe play with a bondage and discipline language to expose yourself to the concepts the parent comment is advocating for.
goatinaboat 6y ago
However, to me R appears like a little better Swiss Army Knife to do initial analysis. ggplot2, tidyverse, ... R is far superior for interactive exploration/analysis and report writing. However Python is far superior if you are writing a program that does other things too. My rule of thumb is that if a Python program is 70% or more Numpy/Pandas/Matplotlib etc then it should be R. Whereas an R program does comparatively little analysis and a lot of logic and integration, it should be Python. No one size fits all.
- kzrdude 6y ago
  Can you "import R" in python? Sounds like that would be best.
  - goatinaboat 6y ago
    Call Python from R: https://cran.r-project.org/web/packages/reticulate/vignettes/calling_python.html https://cran.r-project.org/web/packages/reticulate/vignettes... Calling R from Python: https://pypi.org/project/rpy2/ https://pypi.org/project/rpy2/ I'm not sure I'd ever mix the two directly myself; I'd compose my application as separate R and Python communicating somehow. It seems cleaner.
    - kzrdude 6y ago
      Haha, what you said made me think: sagemath! And sure enough it has something to tie in R too, truly connecting everything in a big nest: https://doc.sagemath.org/html/en/reference/interfaces/sage/interfaces/r.html https://doc.sagemath.org/html/en/reference/interfaces/sage/i...
coldtea 6y ago
>How popular is R in general? Very popular. To the point of even having quite a lot of Microsoft support, lots of books, etc.
williamstein 6y ago
For what it is worth (not at all clear), TIOBE ranked R as the 9th most popular programming language in the world this month: https://www.tiobe.com/tiobe-index/ https://www.tiobe.com/tiobe-index/. For comparison, Python is ranked number 2.
sbassi 6y ago
In finance and fintech is pretty standard.
bayeslaw 6y ago
no self respecting person who calls themselves a date scientist ai researcher or ml engineer would touch R.. it's a toy for making pretty plots and fitting traditional stats models to small data.. it is not a proper programming language but a horrible old scripting whatever that was unfortunately saved by Hadley and his persistence on creating an eco system around it..
- disgruntledphd2 6y ago
  So I guess the authors of the Elements of Statistical Learning aren't "real" researchers then? For reference, the authors of that book (the best book about ML in general) were all involved in the development of S and R.
- data_ders 6y ago
  clicked on link expecting R shitposting. was not disappointed.

kickout 6y ago

Probably the most popular non-software engineer language for working with data. Millions and millions of users that have no idea what this blog post is technically about (but is interesting nonetheless)

NegativeLatency 6y ago
Second after Excel?
- deleted 6y ago
  [deleted]
- kickout 6y ago
  Fair, although I would say Excel and R are serving two separate purposes. But yes, Excel is of course #1
  - wtetzner 6y ago
    > Fair, although I would say Excel and R are serving two separate purposes. It's true that they target different use cases overall (obviously with some overlap), but Excel tends to be used for lots of things that would be better handled with a different tool, because it's what people know.
ISL 6y ago
Matlab has a substantial userbase, too.
- kickout 6y ago
  Agreed, but IMO R >> MatLab and SAS over the past ~10 years. Both Matlab (physics??) and SAS (pharma/financial) seem to have further sunk into deep niches.
  - mjn 6y ago
    Matlab is still big in engineering as well. Matlab+Simulink in particular seems to have a fairly entrenched niche.
ImaCake 6y ago
I think this nails it. Given both python and R, these people will pick R because the motivation for using R is very clear, it does data analysis and statistics. Whereas python kinda does everything and that makes it a bit more tricky to understand.

J253 6y ago

In my experience I’ve seen R used in more exploratory/ad hoc type analysis and algorithm development by “non-developers”—-statisticians, scientists, etc. usually without performance consideration—-and that code is then turned into production code with the dev team using Python or C or something more performant or maintainable.

pyromine 6y ago
R is a nasty nasty language for productionalizing things, honestly it's just too flexible and let's you do the craziest things. But being so flexible makes it really expressive for doing ad-hoc analysis where you really don't know what you're looking for yet.
- laichzeit0 6y ago
  It’s not just the language/runtime though. It’s the entire ecosystem around that that’s required to productionize it in any modern sense of the word. It’s hard to get right even in Python. I mean Flask (even with restx) still doesn’t even generate Swagger 3 documentation.

jhfdbkofdcho 6y ago

Extremely popular and widely used. Pandas etc are Python implementations of R constructs

ekianjo 6y ago

Very popular in academia, moderately popular in industry when it comes to data science/analysis. In any case, very powerful while Python has certainly numerous advantages over it.

vmchale 6y ago

Lots of scientists like it, psychologists and statisticians and such.

FranzFerdiNaN 6y ago

I work with people who mostly have a background in the social sciences or humanities and who work in R pretty much every day. They dont see themselves as programmers and Python is complete gibberish for them, while R just makes sense. When i meet people from other companies in roughly the same space (i work in healthcare doing data analysis), it's mostly the same. I actually meet more people who use SAS/SPSS than Python. For data analysis, R is in my opinion better than Python. It's when you have to integrate it in existing workflows that Python quickly becomes a better choice.

lottin 6y ago
It's not so much that Python is gibberish but that is written, as far as I know, predominantly by engineers, who aren't really experts in statistics, or experts in science for that matter. A scientist will tend to trust more code written by another scientist than code written by an engineer. At least, I would.
- milesvp 6y ago
  I find this statement interesting. Historically scientists have a reputation for writing relatively poor code. Code that runs really slowly due to things like unintended nested loops, or striding values (x,y vs y,x). And code that doesn't handle non-happy path cases very well. Are you saying that you trust the code more because the domain knowledge make it more likely to get the right answer then? Has general knowledge increased such that scientists' code isn't as painful as it was 20 years ago?
  - lottin 6y ago
    Yeah, exactly, even though it might be terribly inefficient, I still trust scientific code more when it's written by scientists than when it's written by non-scientists, in terms of getting the right answer.
bigger_cheese 6y ago
Similar story for me. I am an Engineer (the non software type). I work at an industrial plant. We use SAS pretty extensively for data analysis, time series analysis, multivariate regressions etc. As well as for BI type stuff (reports, graphing, adhoc queries). For a while R was being pushed pretty heavily as a SAS alternative. My org paid R training courses etc. I found R and SAS pretty comparable at least the R packages we looked at (dpylr, ggplot2 etc). I know about Python, the programming language I used PyGTK back in the day to build GUI apps. But it would not be my first thought for doing data analysis work. Does Python even offer something like R studio/ SAS Enterprise Guide and does it have a trending package?

minimaxir 6y ago

As a data scientist who is proficient in both Python and R ecosystems, in my opinion R/tidyverse is substantially better for ad hoc EDA and data visualization. However, Python is better for nearly everything else in the field (namely, working with nontabular data, external APIs, deep learning, and productionization). It's about knowing which tool to use.

omarhaneef 6y ago
Ditto (not that I am proficient, but my experience matches). However, because the rest is easier in python, and my mental gears grind when I switch from one to the other, I end up using Python for the adhoc EDA and viz, and with Spyder, it is a pretty decent experience.
disgruntledphd2 6y ago
> (namely, working with nontabular data, external APIs, deep learning, and productionization). I agree with all of that except for productionisation. I would have agreed before dealing with issues around getting consistent versions of python + libraries to run. The issues I see with Python are as follows: - pip doesn't actually check to make sure your dependencies are compatible, which causes real problems with numpy et al - conda isn't available by default, and running it on remote boxes is non-trivial (I spent a whole week figuring out how to get it running in a remote non-login shell). - This makes it really, really difficult to actually get a standard set of libraries to depend upon, which is really important for production. R, on the other hand, actually resolves dependencies in its package manager, and the R CMD BUILD for packages, while super annoying helps you produce (more) portable code (did you know that conda doesn't provide cross-platform yml files unless invoked specifically?). In terms of handing it over to engineering/non data science people though, Python is much much much better. tl;dr Python's an ace language with a terrible production story.
- ekianjo 6y ago
  > R, on the other hand, actually resolves dependencies in its package manager And you have packages like renv which also help isolate specific versions of packages to make portable environments even more reliable.

ryanar 6y ago

I love using R for exploratory work. Hadley Wickham's TidyVerse of packages make everything so ergonomic to use.

currymj 6y ago

think of it like shell scripting for statistics, although not nearly as limited as bash is compared to other programming languages. it works best if it's used semi-interactively, as a glue language between statistical packages which may be written in other languages. or to write simple "batch" scripts that basically just run a bunch of procedures in a row. RStudio makes the whole experience much nicer in terms of plotting, and RMarkdown is great for preparing documents. of course like shell scripting you can write fairly complicated programs in it, and sometimes people do, but due to backwards compatibility and weird design choices meant to make interactive use easier, programming "in the large" can get weird. the analogy works for Python too -- it is definitely reasonable to use Python for shell scripting, but using Python interactively to pipe things from one program to another is slightly more frustrating than doing it in the shell, although might be preferred due to its other advantages.

totalperspectiv 6y ago

More popular than I wish it was. It is the bash of the data science world. Totally ubiquitous and kind of a dumpster fire.

dcolkitt 6y ago
I disagree, the language is extremely powerful for interactive data exploration. A terse one-liner is all it takes to compute something like "what's the correlation between number of children and home size for people over 45 who live in counties with income variance at the 90th percentile weighted by population". Not that pandas/scipy/numpy don't make an admirable job. You can do something like this, but it's nowhere near as ergonomic as it is R. At the end of the day, R is fundamentally a language for data exploration, whereas with python those facilities are bolted on top of a general purpose environment.
tharne 6y ago
This is the best description of R that I've ever come across, and I say that as someone who learned R as their first programming language. Big mistake, btw. It took me years to unlearn all of the terrible habits I picked up from the R world. Do yourself a favor and start with python, if only to learn proper programming practices and techniques before diving into R.
- uomopertica 6y ago
  While I agree that Python is better than R for programming etiquette, I would argue that proper programming practices and techniques are better learned in languages with static typing and proper variable scoping. Do yourself another favor and also look into C#, Swift or even Java.
  - coward8675309 6y ago
    If R is a reasonable tool for a given problem, C# or Swift or Java almost certainly will not be. The realistic alternatives to R are other numerical analysis packages, Julia, and Python. “The” answer for any given person or project is likely to be a function of your colleagues and peer group, your problem domain, your library needs. One of course is allowed to learn more than one thing. Maybe play with a bondage and discipline language to expose yourself to the concepts the parent comment is advocating for.

goatinaboat 6y ago

However, to me R appears like a little better Swiss Army Knife to do initial analysis. ggplot2, tidyverse, ... R is far superior for interactive exploration/analysis and report writing. However Python is far superior if you are writing a program that does other things too. My rule of thumb is that if a Python program is 70% or more Numpy/Pandas/Matplotlib etc then it should be R. Whereas an R program does comparatively little analysis and a lot of logic and integration, it should be Python. No one size fits all.

kzrdude 6y ago
Can you "import R" in python? Sounds like that would be best.
- goatinaboat 6y ago
  Call Python from R: https://cran.r-project.org/web/packages/reticulate/vignettes/calling_python.html https://cran.r-project.org/web/packages/reticulate/vignettes... Calling R from Python: https://pypi.org/project/rpy2/ https://pypi.org/project/rpy2/ I'm not sure I'd ever mix the two directly myself; I'd compose my application as separate R and Python communicating somehow. It seems cleaner.
  - kzrdude 6y ago
    Haha, what you said made me think: sagemath! And sure enough it has something to tie in R too, truly connecting everything in a big nest: https://doc.sagemath.org/html/en/reference/interfaces/sage/interfaces/r.html https://doc.sagemath.org/html/en/reference/interfaces/sage/i...

coldtea 6y ago

>How popular is R in general? Very popular. To the point of even having quite a lot of Microsoft support, lots of books, etc.

williamstein 6y ago

For what it is worth (not at all clear), TIOBE ranked R as the 9th most popular programming language in the world this month: https://www.tiobe.com/tiobe-index/ https://www.tiobe.com/tiobe-index/. For comparison, Python is ranked number 2.

sbassi 6y ago

In finance and fintech is pretty standard.

bayeslaw 6y ago

no self respecting person who calls themselves a date scientist ai researcher or ml engineer would touch R.. it's a toy for making pretty plots and fitting traditional stats models to small data.. it is not a proper programming language but a horrible old scripting whatever that was unfortunately saved by Hadley and his persistence on creating an eco system around it..

disgruntledphd2 6y ago
So I guess the authors of the Elements of Statistical Learning aren't "real" researchers then? For reference, the authors of that book (the best book about ML in general) were all involved in the development of S and R.
data_ders 6y ago
clicked on link expecting R shitposting. was not disappointed.