5 ms·

> But an alternative often doesn't exist. Python, Julia, Matlab, Java, Scala don't exist for doing what? Are there specific R libraries and functions that aren

> But an alternative often doesn't exist. Python, Julia, Matlab, Java, Scala don't exist for doing what? Are there specific R libraries and functions that aren't reproduced in other languages?

singingfish 9y ago
Yes, R is for statistical analysis. It's absolutely great for that (similar in other languages will take 10-100x more LoC with all the possibilities for bugs it implies). On the other hand it's an absolutely awful general purpose language.
- goatlover 9y ago
  I know R is great for that, but are Python stats libraries lacking to the point of needing 10-100x more code? My understanding is that Python has caught up with R in the data science field in terms of popularity and usage. You can also use R inside Python using the rpy2 library. And Julia embeds R with the RCall package.
  - disgruntledphd2 9y ago
    It really hasn't. Python has pretty good support for machine learning on data with no missing values. It doesn't really support things like survival analysis, complicated time series models or (the most important) missing data. Also, R's formula interface for models is a thing of beauty, while Python has nothing similar. So, if you have data with no missings and are only interested in a small subset of statistics (that used by computer scientists), then Python can definitely suffice. If you are in almost any other discipline, you're going to be using R. Don't get me wrong, Python is a wonderful language, but R is so well adapted to analysis that it will be a very long time before it gets replaced (Julia looks really promising, but its not at 1.0 yet, and hence hasn't seen the broad scale adoption it would need).
    - wrinkl3 9y ago
      So why has nobody tried to implement something similar for Python? Is there some fundamental limitation of the language at play?
      - ForHackernews 9y ago
        Academic statisticians know and use R. They publish research in R. Python doesn't have the same mindshare, so somebody would have to specifically go out and re-implement it in Python. So far, it seems like nobody has been sufficiently motivated to do so.
      - singingfish 9y ago
        You'd basically be implementing a new interpreter (and presumably trying to do so in python). And hauling in a bunch of Fortran bindings along for the ride.
    - singingfish 9y ago
      It's not just that. R has a few things going for it: 1. It's a functional language, and functional programming style lends itself really well to matrix operations (and the engine of stats is all about matrices). 2. Matrices are baked really deeply into the language, which makes the functional style even more powerful. (my understanding is that what makes the formula interface work so well). 3. R can be viewed as coating of syntactic sugar on a lot of very useful fortran libraries. On the other hand I once saw some open source toolkit for natural language processing in R. I had two issues with that. Firstly WTF was the author thinking doing all his general purpose programming in R. And two I didn't want to investigate further because as NLP is largely about general purpose operations and I didn't want to get sucked into that rabbit hole.
    - jkabrg 9y ago
      One thing I don't like about Python libraries, from the top of my head: That there is a distinction between functions and methods, e.g. between nansum(a) and a.sum() where a is a numpy array. The distinction is illogical and just has to be memorized, much like the difference between different categories of verbs in French. In contrast, R uses an exotic form of object oriented programming called multimethods, which doesn't make a distinction between object.method() and method(object). And if you like chaining methods together like object.method1().method2().method3(), then you can do that in R using the piping operator from the Dplyr package: object %>% method1() %>% method2() %>% method3(), which is rightly equivalent to method3(method2(method1(object)))
  - hadley 9y ago
    Things (IMO) that R has going for it: * RStudio IDE - none of python equivalents are anywhere near as mature * Functional programming - I can't fully articulate why yet, but to me, FP seems like it "fits" most DS programming challenges better than OO. * Tidyverse - I'm biased (being the author) but having a broad ecosystem of tools with shared underlying programming philosophies makes solving data science challenges particularly easy/painless/fun. * RMarkdown - makes it so easy to intermingle prose + code and then (via pandoc) render to a very wide variety of outputs (html, word, pdf, html slides, dashboards, ...) * Shiny - for cases where you don't want a single report but an app, an R user can quickly create an interactive app without knowing the details of html/css/js * Very broad ecosystem of statistical packages. Most statistics researchers use R, so you're more like to see cutting edge stat research in R first.
- hadley 9y ago
  It's not a terrible general purpose language (unless you believe that only OO languages can be general purpose). It could definitely do with more general purpose programming packages, but that's not a limitation of the language.
ACow_Adonis 9y ago
Aside from the libraries issues... Understand I'm talking about in the context of my workplaces and experiences, which, also in my experience, are a significant number of employers hiring data science-type jobs. An alternative doesn't exist in the sense that we've been battling for years to get things that are more open and aren't SAS/Matlab/SPSS. Finally we're at the stage where something is finally getting semi-approved and installed reliably and the doors are opening...and its...R. Python is becoming more common. Admittedly. And kudos to anaconda for its non-admin requiring install. But in some ways even installing that is a bit "naughty". But Julia, Java, Scala, C? No no. Those are programming languages. Software companies get those. Maybe a handful of guys in IT if they need it for some reason, but many are still working in places where most IT/security staff can't tell the difference between R and RStudio. Github is blocked by the firewall and security :P So the idea of options...maybe in another 10 years :P Until then, we take what we can get...
- owlninja 9y ago
  What can R do that Python can't right now?
- Chris2048 9y ago
  Julia seems similar to python, and even runs fine on Jupyter. I think https://data-sorcery.org/ https://data-sorcery.org/ (incanter) might be good JVM environment.
wakkaflokka 9y ago
There are some things that still aren't available in Python unfortunately. For example, there isn't a package that I'm aware of that will do a mixed-effects logistic regression in Python.

> But an alternative often doesn't exist. Python, Julia, Matlab, Java, Scala don't exist for doing what? Are there specific R libraries and functions that aren't reproduced in other languages?

singingfish 9y ago

Yes, R is for statistical analysis. It's absolutely great for that (similar in other languages will take 10-100x more LoC with all the possibilities for bugs it implies). On the other hand it's an absolutely awful general purpose language.

goatlover 9y ago
I know R is great for that, but are Python stats libraries lacking to the point of needing 10-100x more code? My understanding is that Python has caught up with R in the data science field in terms of popularity and usage. You can also use R inside Python using the rpy2 library. And Julia embeds R with the RCall package.
- disgruntledphd2 9y ago
  It really hasn't. Python has pretty good support for machine learning on data with no missing values. It doesn't really support things like survival analysis, complicated time series models or (the most important) missing data. Also, R's formula interface for models is a thing of beauty, while Python has nothing similar. So, if you have data with no missings and are only interested in a small subset of statistics (that used by computer scientists), then Python can definitely suffice. If you are in almost any other discipline, you're going to be using R. Don't get me wrong, Python is a wonderful language, but R is so well adapted to analysis that it will be a very long time before it gets replaced (Julia looks really promising, but its not at 1.0 yet, and hence hasn't seen the broad scale adoption it would need).
  - wrinkl3 9y ago
    So why has nobody tried to implement something similar for Python? Is there some fundamental limitation of the language at play?
    - ForHackernews 9y ago
      Academic statisticians know and use R. They publish research in R. Python doesn't have the same mindshare, so somebody would have to specifically go out and re-implement it in Python. So far, it seems like nobody has been sufficiently motivated to do so.
    - singingfish 9y ago
      You'd basically be implementing a new interpreter (and presumably trying to do so in python). And hauling in a bunch of Fortran bindings along for the ride.
  - singingfish 9y ago
    It's not just that. R has a few things going for it: 1. It's a functional language, and functional programming style lends itself really well to matrix operations (and the engine of stats is all about matrices). 2. Matrices are baked really deeply into the language, which makes the functional style even more powerful. (my understanding is that what makes the formula interface work so well). 3. R can be viewed as coating of syntactic sugar on a lot of very useful fortran libraries. On the other hand I once saw some open source toolkit for natural language processing in R. I had two issues with that. Firstly WTF was the author thinking doing all his general purpose programming in R. And two I didn't want to investigate further because as NLP is largely about general purpose operations and I didn't want to get sucked into that rabbit hole.
  - jkabrg 9y ago
    One thing I don't like about Python libraries, from the top of my head: That there is a distinction between functions and methods, e.g. between nansum(a) and a.sum() where a is a numpy array. The distinction is illogical and just has to be memorized, much like the difference between different categories of verbs in French. In contrast, R uses an exotic form of object oriented programming called multimethods, which doesn't make a distinction between object.method() and method(object). And if you like chaining methods together like object.method1().method2().method3(), then you can do that in R using the piping operator from the Dplyr package: object %>% method1() %>% method2() %>% method3(), which is rightly equivalent to method3(method2(method1(object)))
- hadley 9y ago
  Things (IMO) that R has going for it: * RStudio IDE - none of python equivalents are anywhere near as mature * Functional programming - I can't fully articulate why yet, but to me, FP seems like it "fits" most DS programming challenges better than OO. * Tidyverse - I'm biased (being the author) but having a broad ecosystem of tools with shared underlying programming philosophies makes solving data science challenges particularly easy/painless/fun. * RMarkdown - makes it so easy to intermingle prose + code and then (via pandoc) render to a very wide variety of outputs (html, word, pdf, html slides, dashboards, ...) * Shiny - for cases where you don't want a single report but an app, an R user can quickly create an interactive app without knowing the details of html/css/js * Very broad ecosystem of statistical packages. Most statistics researchers use R, so you're more like to see cutting edge stat research in R first.
hadley 9y ago
It's not a terrible general purpose language (unless you believe that only OO languages can be general purpose). It could definitely do with more general purpose programming packages, but that's not a limitation of the language.

ACow_Adonis 9y ago

Aside from the libraries issues... Understand I'm talking about in the context of my workplaces and experiences, which, also in my experience, are a significant number of employers hiring data science-type jobs. An alternative doesn't exist in the sense that we've been battling for years to get things that are more open and aren't SAS/Matlab/SPSS. Finally we're at the stage where something is finally getting semi-approved and installed reliably and the doors are opening...and its...R. Python is becoming more common. Admittedly. And kudos to anaconda for its non-admin requiring install. But in some ways even installing that is a bit "naughty". But Julia, Java, Scala, C? No no. Those are programming languages. Software companies get those. Maybe a handful of guys in IT if they need it for some reason, but many are still working in places where most IT/security staff can't tell the difference between R and RStudio. Github is blocked by the firewall and security :P So the idea of options...maybe in another 10 years :P Until then, we take what we can get...

owlninja 9y ago
What can R do that Python can't right now?
Chris2048 9y ago
Julia seems similar to python, and even runs fine on Jupyter. I think https://data-sorcery.org/ https://data-sorcery.org/ (incanter) might be good JVM environment.

wakkaflokka 9y ago

There are some things that still aren't available in Python unfortunately. For example, there isn't a package that I'm aware of that will do a mixed-effects logistic regression in Python.