7 ms·

I sometimes wonder is there any reason to learn R at all, since python eco system has absorbed most of its advanced statistical functionality, coupled with the

I sometimes wonder is there any reason to learn R at all, since python eco system has absorbed most of its advanced statistical functionality, coupled with the factor that python environment is much more general, with capabilities to fetch, decode/encoded data,work with binary data databases, web frameworks for presenting etc.

anthony_doan 8y ago
> since python eco system has absorbed most of its advanced statistical functionality This isn't true at all... Also all advance statistical books are either SAS or R. If it's R then there is always a package that the author created. Just look at Chapman & Hall/CRC or Springer publisher and look at their books. Go here: https://www.jstatsoft.org/index https://www.jstatsoft.org/index Count the number of R packages in those papers versus Python. I don't even need a source. I'm a statistician and I'm going to get a paper there and publish a R package for my master thesis.
minimaxir 8y ago
I use both Python and R. tidyverse/ggplot2 alone are enough reason to use R, and are substantially faster for tasks that utilize those packages than the equivalent in Python (in my opinion). Although I haven't had as much reason to use base R. For more ML-related tasks I do go back to Python.
- logjammin 8y ago
  Seconded on all points. I do branch out to SQL for stuff, too, and I find that R and Python play nicely with it, too. But as long as ggplot exists and Python doesn't have it, R will never really leave my side.
- jwilbs 8y ago
  This. I’ve contributed code to popular libraries in both languages, and while I (overall) have a preference for python (mostly due to it being general purpose), I find R code unparalleled when it comes to raw data manipulation/analysis. The overall api of tidyverse packages is such a joy, and recent improvements in purrr/tidyr allow me to construct nested data analysis workflows I couldn’t even dream of in python.
  - marmaduke 8y ago
    Would you have an example of that?
  - ppod 8y ago
    One random example I found recently is a tidyverse package called forcats that has lots of nice functions for categorical data. For example, it has a single function that merges all categories with a frequency of less than a certain threshold in the table into a new category like "other" or whatever. This is a task I often need to do, but as far as I can see it's a bit of a hack in python or pandas. It's just lots of little things like this, especially wrangling data tables. https://forcats.tidyverse.org/reference/fct_lump.html https://forcats.tidyverse.org/reference/fct_lump.html There's also the data.table package for this kind of data work, which is maybe less used but seems to have better performance.
- roenxi 8y ago
  Here here. Tidyverse also provides a centralised 'this is how you do X' nexus really helps discover-ability. World class stuff, on tap. For example, I know the recommended pipe in R is magrittr's %>%. I have no idea what the respectable pipe library in Python is, or even if there is one. I wouldn't even know where to start finding all the tidyverse equivalents in Python. It isn't as organised and obvious as the R statistics community. On the other hand Base R is the worst. Disgusting language.
  - spectramax 8y ago
    Julia has a |> operator and it works amazingly well with Queryverse.jl which is a clone of Tidyverse!
- IanCal 8y ago
  I'm finding a fairly nice combo is using rmarkdown, python and reticulate to do the things that are easier in python there and the outputs in R. Debugging isn't where I'd like it to be yet but there might be a way of improving that - I haven't explored yet.
Canadauni 8y ago
I use mixed effects models pretty extensively. While there is an implementation in statsmodels the implementation in lme4 is more user friendly and has a more mature ecosystem of post-hoc tests.
j7ake 8y ago
Is there anything comparable to tidyverse and ggplot2 in python? If so I will switch immediately.
- groceryheist 8y ago
  To answer your question: ggplot2 : plotnine is quite good ggplot2 clone based on matplotlib. I feel like ggplot2 is a bit better and more complete, but if you want to do something that isn't supported it's harder for me to hack than matplotlib. Tidyverse: To me, ggplot2 is the only essential part of the tidyverse. Lubridate is also good. Most others seem like semantics and syntax sugar. I prefer data.table, which is similar to Pandas. DT is super fast but imho Pandas has a more intuitive and consistent API (and if you want a speed up for large N then dask might work). I use both R and Python on a regular basis. I choose Python for lower-level stuff, automation, parallelism / concurrency, and R for bespoke statistics. I use both for everyday statistics and plotting, but I feel that R has light advantages. I feel like if you're comfortable switching languages there are good reasons to use both. It's also important for me because I work with different teams that have different practices and preferences.
kuzehanka 8y ago
I don't think there's any reason to learn R for anyone who is already proficient at programming. Despite being proficient with R, the only times I used it in the last two years were for ggplot. And even for data vis, I'm increasingly using Python and JS. There's a bunch of comments below which can be summed up with 'use R because <package name> doesn't have a direct python equivalent' but they're all missing the point that the Python data science ecosystem is evolving at a much faster pace than R and will completely supersede it in a few years. R, like SAS, is a tool for non-programmers. And there it shall remain. The only demographic where R makes sense long term are pure mathematicians/statisticians who are not proficient in programming. But that demographic is rapidly declining in size.
- anthony_doan 8y ago
  > There's a bunch of comments below which can be summed up with 'use R because <package name> doesn't have a direct python equivalent' but they're all missing the point that the Python data science ecosystem is evolving at a much faster pace than R and will completely supersede it in a few years. The point is R is a very good language for statistic because of the packages not data science. Data science can do their own thing it's okay. It's also okay for data science to use statistic models from statistic too. > R, like SAS, is a tool for non-programmers. I respect and love data science and machine learning but this behavior of generalization is terrible. There are many wondeful programmers contribute to R and uses R as I am sure there are many wonderful statisticians that use Python. They're just tools. > And there it shall remain. The only demographic where R makes sense long term are pure mathematicians/statisticians who are not proficient in programming. But that demographic is rapidly declining in size. What is up with these generalizations? R is not going anywhere in the statistic community. It's doing fine. Also from my experiences in academia most math people use matlab and if any R. It's okay to have both R and Python doing their thing. There is no need to conflate data science and statistic or have this weird tribalism.
  - kuzehanka 8y ago
    Everything you said sums up with 'R is a very good language for statistic because of the packages' which is pretty much in agreement with the GP comment. R has nothing going for it except a rapidly dwindling number of packages that don't yet have a direct python equivalent. It doesn't make sense to invest time into R if one already knows python unless one specifically focusing on academia pure stats type stuff. Even then, the incoming generation of undergrads are increasingly proficient with programming and are shying away from R the same way that they shied away from Matlab after scipy matched it for 95% of their tasks.
mikorym 8y ago
I use both python and R almost every day. Although I like R and often use R to quickly order tabulated data, there are a few things to take into account that in recent times are building a strong case for me not to use R habitually. Development in R is frustrating. If you don't need to do dev, then on this point you are home free. Testing things that you deploy in R is not simple. Scripting in R can be frustrating. I have a script that traverses Excel files and using tryCatch() is just so much more complicated with it being a function. In Python the try-catch functionality is part of the design syntax. There are scenarios where R is better. If you are in actuarial science, research or academics then often you'll find R libraries that just work. R treats tabular data with grace. Everything in R is an array. The takeaway for me is that I should use R less and Python more. I personally can't deal with something like tryCatch() being overcomplicated, but for people who don't do dev anyway and maybe need to analyse DNA sequences for a living, R can be rewarding. For me: the ggplot2 library is great; stay away from Shiny and dev in R.
- mh12345 8y ago
  Interesting, why do you advise people to stay away from Shiny?
  - mikorym 8y ago
    It tries to do html, but it is limited. So I'd rather use Javascript to manipulate the frontend directly. It tries to do functional programming, but the documentation is not satisfying. The responses and behaviour is perplexing. I spent around 5—10 hours trying to get a Shiny GUI to work and eventually got to the conclusion that 1) if you want a big project do all the frontend stuff in something else, like JS and 2) if you want a small project try something established (I am not advocating, it's just an example) like Power BI.
    - mh12345 8y ago
      Regarding the limited frontend capabilities, I had a similar opinion at one point, but with Shiny's HTML templating (https://shiny.rstudio.com/articles/templates.html https://shiny.rstudio.com/articles/templates.html) functionality one can circumvent the limited HTML that Shiny has out of the box. Besides that, there is also the possibility to communicate with R using JavaScript (https://shiny.rstudio.com/articles/communicating-with-js.html https://shiny.rstudio.com/articles/communicating-with-js.htm...). These two functionalities combined allow for a frontend that is much more flexible, when compared to traditional Shiny applications. Of course, there might definitely be better solutions out there that fit your use case and Shiny's real use is primarily in sharing data analysis.
arendtio 8y ago
I don't know if it is still a thing, but if you are working with SAP HANA (in-memory database) there is a good chance you would like to learn R as they integrated it into their database.
- Annatar 8y ago
  Vertica did as well.
- hdkrgr 8y ago
  Related: tidyverse's dbplyr let's you write tidyverse code querying almost any remote database - leveraging the DBs computation while writing code (almost) as you would for a local data frame. In my old job I got to a point where I would barely ever need to write sql anymore because of this. https://cran.r-project.org/web/packages/dbplyr/vignettes/dbplyr.html https://cran.r-project.org/web/packages/dbplyr/vignettes/dbp...
Bootvis 8y ago
All those general things you can do in R as well. Maybe not as well developed and widely used as the Python counterparts but definitily there.
jhbadger 8y ago
Not really in my experience. Really, the only place where I'd say Python has gotten more support so far than R is in deep learning. If you want any just-published statistical method, the associated implementation will almost inevitably be in R. But that's today -- I'm old enough to remember when the standard language in "The Journal of Statistical Software" was XLISP-STAT (much of the 1990s).

anthony_doan 8y ago

> since python eco system has absorbed most of its advanced statistical functionality This isn't true at all... Also all advance statistical books are either SAS or R. If it's R then there is always a package that the author created. Just look at Chapman & Hall/CRC or Springer publisher and look at their books. Go here: https://www.jstatsoft.org/index https://www.jstatsoft.org/index Count the number of R packages in those papers versus Python. I don't even need a source. I'm a statistician and I'm going to get a paper there and publish a R package for my master thesis.

minimaxir 8y ago

I use both Python and R. tidyverse/ggplot2 alone are enough reason to use R, and are substantially faster for tasks that utilize those packages than the equivalent in Python (in my opinion). Although I haven't had as much reason to use base R. For more ML-related tasks I do go back to Python.

logjammin 8y ago
Seconded on all points. I do branch out to SQL for stuff, too, and I find that R and Python play nicely with it, too. But as long as ggplot exists and Python doesn't have it, R will never really leave my side.
jwilbs 8y ago
This. I’ve contributed code to popular libraries in both languages, and while I (overall) have a preference for python (mostly due to it being general purpose), I find R code unparalleled when it comes to raw data manipulation/analysis. The overall api of tidyverse packages is such a joy, and recent improvements in purrr/tidyr allow me to construct nested data analysis workflows I couldn’t even dream of in python.
- marmaduke 8y ago
  Would you have an example of that?
- ppod 8y ago
  One random example I found recently is a tidyverse package called forcats that has lots of nice functions for categorical data. For example, it has a single function that merges all categories with a frequency of less than a certain threshold in the table into a new category like "other" or whatever. This is a task I often need to do, but as far as I can see it's a bit of a hack in python or pandas. It's just lots of little things like this, especially wrangling data tables. https://forcats.tidyverse.org/reference/fct_lump.html https://forcats.tidyverse.org/reference/fct_lump.html There's also the data.table package for this kind of data work, which is maybe less used but seems to have better performance.
roenxi 8y ago
Here here. Tidyverse also provides a centralised 'this is how you do X' nexus really helps discover-ability. World class stuff, on tap. For example, I know the recommended pipe in R is magrittr's %>%. I have no idea what the respectable pipe library in Python is, or even if there is one. I wouldn't even know where to start finding all the tidyverse equivalents in Python. It isn't as organised and obvious as the R statistics community. On the other hand Base R is the worst. Disgusting language.
- spectramax 8y ago
  Julia has a |> operator and it works amazingly well with Queryverse.jl which is a clone of Tidyverse!
IanCal 8y ago
I'm finding a fairly nice combo is using rmarkdown, python and reticulate to do the things that are easier in python there and the outputs in R. Debugging isn't where I'd like it to be yet but there might be a way of improving that - I haven't explored yet.

Canadauni 8y ago

I use mixed effects models pretty extensively. While there is an implementation in statsmodels the implementation in lme4 is more user friendly and has a more mature ecosystem of post-hoc tests.

j7ake 8y ago

Is there anything comparable to tidyverse and ggplot2 in python? If so I will switch immediately.

groceryheist 8y ago
To answer your question: ggplot2 : plotnine is quite good ggplot2 clone based on matplotlib. I feel like ggplot2 is a bit better and more complete, but if you want to do something that isn't supported it's harder for me to hack than matplotlib. Tidyverse: To me, ggplot2 is the only essential part of the tidyverse. Lubridate is also good. Most others seem like semantics and syntax sugar. I prefer data.table, which is similar to Pandas. DT is super fast but imho Pandas has a more intuitive and consistent API (and if you want a speed up for large N then dask might work). I use both R and Python on a regular basis. I choose Python for lower-level stuff, automation, parallelism / concurrency, and R for bespoke statistics. I use both for everyday statistics and plotting, but I feel that R has light advantages. I feel like if you're comfortable switching languages there are good reasons to use both. It's also important for me because I work with different teams that have different practices and preferences.

kuzehanka 8y ago

I don't think there's any reason to learn R for anyone who is already proficient at programming. Despite being proficient with R, the only times I used it in the last two years were for ggplot. And even for data vis, I'm increasingly using Python and JS. There's a bunch of comments below which can be summed up with 'use R because <package name> doesn't have a direct python equivalent' but they're all missing the point that the Python data science ecosystem is evolving at a much faster pace than R and will completely supersede it in a few years. R, like SAS, is a tool for non-programmers. And there it shall remain. The only demographic where R makes sense long term are pure mathematicians/statisticians who are not proficient in programming. But that demographic is rapidly declining in size.

anthony_doan 8y ago
> There's a bunch of comments below which can be summed up with 'use R because <package name> doesn't have a direct python equivalent' but they're all missing the point that the Python data science ecosystem is evolving at a much faster pace than R and will completely supersede it in a few years. The point is R is a very good language for statistic because of the packages not data science. Data science can do their own thing it's okay. It's also okay for data science to use statistic models from statistic too. > R, like SAS, is a tool for non-programmers. I respect and love data science and machine learning but this behavior of generalization is terrible. There are many wondeful programmers contribute to R and uses R as I am sure there are many wonderful statisticians that use Python. They're just tools. > And there it shall remain. The only demographic where R makes sense long term are pure mathematicians/statisticians who are not proficient in programming. But that demographic is rapidly declining in size. What is up with these generalizations? R is not going anywhere in the statistic community. It's doing fine. Also from my experiences in academia most math people use matlab and if any R. It's okay to have both R and Python doing their thing. There is no need to conflate data science and statistic or have this weird tribalism.
- kuzehanka 8y ago
  Everything you said sums up with 'R is a very good language for statistic because of the packages' which is pretty much in agreement with the GP comment. R has nothing going for it except a rapidly dwindling number of packages that don't yet have a direct python equivalent. It doesn't make sense to invest time into R if one already knows python unless one specifically focusing on academia pure stats type stuff. Even then, the incoming generation of undergrads are increasingly proficient with programming and are shying away from R the same way that they shied away from Matlab after scipy matched it for 95% of their tasks.

mikorym 8y ago

I use both python and R almost every day. Although I like R and often use R to quickly order tabulated data, there are a few things to take into account that in recent times are building a strong case for me not to use R habitually. Development in R is frustrating. If you don't need to do dev, then on this point you are home free. Testing things that you deploy in R is not simple. Scripting in R can be frustrating. I have a script that traverses Excel files and using tryCatch() is just so much more complicated with it being a function. In Python the try-catch functionality is part of the design syntax. There are scenarios where R is better. If you are in actuarial science, research or academics then often you'll find R libraries that just work. R treats tabular data with grace. Everything in R is an array. The takeaway for me is that I should use R less and Python more. I personally can't deal with something like tryCatch() being overcomplicated, but for people who don't do dev anyway and maybe need to analyse DNA sequences for a living, R can be rewarding. For me: the ggplot2 library is great; stay away from Shiny and dev in R.

mh12345 8y ago
Interesting, why do you advise people to stay away from Shiny?
- mikorym 8y ago
  It tries to do html, but it is limited. So I'd rather use Javascript to manipulate the frontend directly. It tries to do functional programming, but the documentation is not satisfying. The responses and behaviour is perplexing. I spent around 5—10 hours trying to get a Shiny GUI to work and eventually got to the conclusion that 1) if you want a big project do all the frontend stuff in something else, like JS and 2) if you want a small project try something established (I am not advocating, it's just an example) like Power BI.
  - mh12345 8y ago
    Regarding the limited frontend capabilities, I had a similar opinion at one point, but with Shiny's HTML templating (https://shiny.rstudio.com/articles/templates.html https://shiny.rstudio.com/articles/templates.html) functionality one can circumvent the limited HTML that Shiny has out of the box. Besides that, there is also the possibility to communicate with R using JavaScript (https://shiny.rstudio.com/articles/communicating-with-js.html https://shiny.rstudio.com/articles/communicating-with-js.htm...). These two functionalities combined allow for a frontend that is much more flexible, when compared to traditional Shiny applications. Of course, there might definitely be better solutions out there that fit your use case and Shiny's real use is primarily in sharing data analysis.

arendtio 8y ago

I don't know if it is still a thing, but if you are working with SAP HANA (in-memory database) there is a good chance you would like to learn R as they integrated it into their database.

Annatar 8y ago
Vertica did as well.
hdkrgr 8y ago
Related: tidyverse's dbplyr let's you write tidyverse code querying almost any remote database - leveraging the DBs computation while writing code (almost) as you would for a local data frame. In my old job I got to a point where I would barely ever need to write sql anymore because of this. https://cran.r-project.org/web/packages/dbplyr/vignettes/dbplyr.html https://cran.r-project.org/web/packages/dbplyr/vignettes/dbp...

Bootvis 8y ago

All those general things you can do in R as well. Maybe not as well developed and widely used as the Python counterparts but definitily there.

jhbadger 8y ago

Not really in my experience. Really, the only place where I'd say Python has gotten more support so far than R is in deep learning. If you want any just-published statistical method, the associated implementation will almost inevitably be in R. But that's today -- I'm old enough to remember when the standard language in "The Journal of Statistical Software" was XLISP-STAT (much of the 1990s).