7 ms·

R is objectively worse than Python for almost all data science tasks, and R is a huge PITA to productionise compared to Python. I've yet to see any argument fo

R is objectively worse than Python for almost all data science tasks, and R is a huge PITA to productionise compared to Python. I've yet to see any argument for R that doesn't boil down to 'well, I know it better' or 'well, I prefer the syntax'. R to data science is as Matlab is to engineering. It's a stopgap 'non programmer' language that thrived at a time when most academics didn't know any programming. Now school children learn programming. There is no use case for these languages anymore.

exdsq 7y ago
> There is no use case for these languages anymore. There are entire ecosystems of academic libraries built around Matlab that can’t all just be picked up and moved to Python. This argument probably doesn’t realise just how ingrained Matlab is in STEM non-CS academic departments. Example: my girlfriends department writes a world-leading MRI analysis library in Matlab. They offer training courses on it (so departments around the world now know it) and it’s frequently used within academic papers (so there are now resources available on it). Why would they move to Python?
- hatmatrix 7y ago
  Also, Simulink - there is no equivalent in Python and many industries use this.
  - exdsq 7y ago
    https://uk.mathworks.com/solutions/aerospace-defense/standards/nasa.html https://uk.mathworks.com/solutions/aerospace-defense/standar... Yep, NASA used it alongside Matlab for the Orion's Guidance and Navigation Control systems. I've never had the chance to use it though, it looks pretty interesting.
- maxnoe 7y ago
  Because more more and more people realize that using closed source, proprietary programming languages and libraries is not compatible with open, reproducible science.
  - exdsq 7y ago
    Sure but in reality most if not all educational institutes that I'm aware of have Matlab licenses, it's what everyone in that particular field uses, and it's better to publish something with Matlab code than nothing at all which is I guess the alternative (it's a means to an end after all). I can imagine this will change in the long run, but right now there are many valid reasons why people use these tools.
- missosoup 7y ago
  > There are entire ecosystems of academic libraries built around Matlab that can’t all just be picked up and moved to Python. They can and they are. Python is increasingly displacing everything in the data industry and especially proprietary legacy platforms like Matlab. The number of things you can do in Matlab but not Python is converging on 0, while the inverse is not even worth trying to count. Major universities are abandoning Matlab, Labview, SPSS, Minitab etc for Python, which is basically the end for them all. The next wave of CS/SE/DS/ML graduates had no exposure to Matlab. It'll linger in electrical engineering for a few more years but will suffer the same fate. In the end, proprietary platforms have no chance against FOSS. > Example: my girlfriends department writes a world-leading MRI analysis library in Matlab Siemens is leading the MRI industry and the only place where they're still using Matlab is the legacy platforms that aren't yet listed for updates or aren't worth updating. The actual leading stuff is done with the same ML tools as the rest of the industry, mostly Tensorflow. Siemens and GE both also have programs to engage and eventually acquire 3rd party ML platforms not a single one of which has anything to do with Labview or Matlab outside of occasionally interfacing with legacy components. https://www.siemens-healthineers.com/en-au/digital-health-solutions/value-themes/artificial-intelligence-in-healthcare https://www.siemens-healthineers.com/en-au/digital-health-so...
  - euler_angles 7y ago
    > Major universities are abandoning Matlab, Labview, SPSS, Minitab etc for Python Just to add another point of anecdata. I helm a large data science effort in the defense industry. We are actively moving away from MATLAB and to Python. It's easier for us to find Python coders, easier to train people to use Python, more maintainable for the restrictions we have on our networks, and cheaper.
- stoops 7y ago
  In python, your MRI analysis library could be trivially hooked up with other cloud data pipelines. Companies would require fewer training courses on average. python is the most popular programming language in the world and getting better.
kensai 7y ago
Than I guess Julia will be an even more powerful language to learn, as it combines the flexibility of Python with all the use cases of all major technical/scientific languages, while being efficient and fast. https://juliacomputing.com/domains/ https://juliacomputing.com/domains/
- bayesian_horse 7y ago
  I think Julia is better designed than R. But it lacks R's ecosystem, so Julia is a tougher sell. I haven't done a lot with Julia, so I don't know if it's easier to teach Julia to novices than Python or R.
  - elcritch 7y ago
    Julia’s ecosystem has been progressing since 1.0. The GLM.jl lib has become much better as has the data frame package. It’s more consistent than Python‘s data-science ecosystem since it’s not tying disparate C code together. But having strong types (actual, not mypy) helps make code more consistent. Still Julia’s ecosystem seems to be building more from R’s more solid academic approach.
    - bayesian_horse 7y ago
      Yes, Julia has a lot of benefits in some regards. The community is great. But small. For a lot of situations, I'd be hesitant to invest in Julia, because I don't know if the community will stay that way or if it fades away.
      - ViralBShah 7y ago
        Out of curiosity, how would you know that the community is large enough, or committed enough? For example, while Julia has been in development for almost 10 years, a lot of the community has now been around for 5 years. There's about 2,500 Julia packages, with the ability to call C, Fortran, R, Python, Java, etc. All the key community stats based on downloads, website views, videos show a healthy growth every year. While in absolute numbers, we may be at 20% of R or Python communities, I am always curious to understand what people mean when they say the community is too small. What would be a signal that a particular community is big enough?
hyperbovine 7y ago
> R is objectively worse than Python for almost all data science tasks If you meant to type "machine learning" I'd probably agree, but R is much much better for small scale data exploration, visualization and modeling (i.e. 95% of DS) than Python. Pandas is an absolute horror show of an API compared to dplyr, and the best plotting libraries for Python are just copying features from R. Lack of a magrittr style infix operator, though seemingly minor, actually emerges as a real pain point once you become accustomed to using it. R is inferior to Python as a programming language, no doubt about it -- but most data scientists are not programmers. Which is the point of TFA.
- missosoup 7y ago
  > but most data scientists are not programmers This is the crux of the problem with R and why R is increasingly blacklisted at large orgs. It attracts non-programmers which may have been okay 5 years ago but is no longer acceptable. With the exception of some engineering powerhouses hiring pure research PhDs to write R code, the trend established over the last 2 years is that fewer and fewer employers are hiring data scientists that aren't programmers. There are too many candidates who know data science and can also do data engineering and even generalist SE tasks. Non-programmer data scientists are not competitive in the industry anymore except that small top-end research niche that doesn't exist in most orgs. Which brings us back to the fact that R was a successful niche language that allowed non-programmers to write models, but that's simply not enough anymore. Businesses want models that can be plugged into production pipelines, models that can scale without needing a dedicated team to re-implement them, and they want staff who do engineering in addition to whatever it is they specialise in. Virtually all data scientists graduating today are programmers, and pretty good ones. Candidates who only know R can't compete against them. > Lack of a magrittr style infix operator, though seemingly minor, actually emerges as a real pain point once you become accustomed to using So you'd agree that you fall into the 'I prefer the syntax' bucket then? I don't really see any arguments against Python in your comment. Funnily enough, it's trivial to implement a pipe style operator in Python and there's at least two popular libraries for that.
  - fock 7y ago
hatmatrix 7y ago
Whenever someone makes a statement that one language is "objectively" better than another, it is followed only by opinions of the author.
FranzFerdiNaN 7y ago
Statisticians, economists, biologists, social scientists all learn and work with R. They publish new packages in R, not in Python. There is no trend at all that this is moving towards Python. Python is far, very far behind when it comes to state of the art research in anything stat-related (besides machine learning i guess, but R is pushing hard to close that gap).
- missosoup 7y ago
  > Statisticians, economists, biologists, social scientists all learn and work with R They used to work with R. And the old generation of engineers used to work with Matlab. The old generation still does. The new generation has been using actual programming languages, typically Python, since high school. They were the first wave of graduates in 2019 that specialised in a discipline and were also competent in software engineering. The old generation is going to be driven out of the job market by the new in the span of 5 years as they saturate the senior tier of their respective fields. How do you compete for a job when all you know is R and your discipline, against someone who's a full fledged software engineer who knows your discipline and can put models directly into production use?
  - bart_spoon 7y ago
    This simply isn't true. There are more of them that know Python these days in addition to R, but as a recent graduate of a respected statistics graduate program, I can assure you that R is still the overwhelmingly preferred choice in the field, and also is in economics.
  - jointpdf 7y ago
    So assuming this is true, what plan of action do you recommend for an “old gen” data scientist who is strong in math, stats, ML theory, R, dataviz, ETL, research, etc., but who is not by a long shot a “full fledged software engineer”? I will soon be competing against this new crop of statistician/engineer superhybrids you speak of. I know a fair bit of Python (mostly for ML/DL applications), bash, and just a smidge of HTML/CSS/JS (just enough tweak a front end demo via R Shiny). I’m OCD enough that I make every effort to write clean and reproducible code and unit test it as I go (is this TDD?). I can implement some stats algorithms (e.g. EM algorithm, MCMC) from scratch with a pseudocode reference, but I rarely if ever have the occasion to do that for obvious reasons. I understand the concept of computational complexity, though I don’t have any figures memorized. But I’ve never taken any CS course beyond Programming 101. I wouldn’t know how to navigate a complex production codebase. Embarrassingly, I know almost nothing about git. I’m 100% sure I’d get slaughtered in a whiteboard interview or similar. For that matter, I could easily get nailed on some holes in my core data science knowledge (cough SQL cough). So, do I rebuild my foundation around software engineering, or just patch up the obvious holes? Grind towards a management position and let my inferior skills rot away?
bart_spoon 7y ago
I would say that for data manipulation and data visualization, R is objectively superior to Python. And for most statistical methods. Its pretty even for many machine learning algorithms. Python only really outclasses R in deep learning in my opinion.
deleted 7y ago
[deleted]

exdsq 7y ago

> There is no use case for these languages anymore. There are entire ecosystems of academic libraries built around Matlab that can’t all just be picked up and moved to Python. This argument probably doesn’t realise just how ingrained Matlab is in STEM non-CS academic departments. Example: my girlfriends department writes a world-leading MRI analysis library in Matlab. They offer training courses on it (so departments around the world now know it) and it’s frequently used within academic papers (so there are now resources available on it). Why would they move to Python?

hatmatrix 7y ago
Also, Simulink - there is no equivalent in Python and many industries use this.
- exdsq 7y ago
  https://uk.mathworks.com/solutions/aerospace-defense/standards/nasa.html https://uk.mathworks.com/solutions/aerospace-defense/standar... Yep, NASA used it alongside Matlab for the Orion's Guidance and Navigation Control systems. I've never had the chance to use it though, it looks pretty interesting.
maxnoe 7y ago
Because more more and more people realize that using closed source, proprietary programming languages and libraries is not compatible with open, reproducible science.
- exdsq 7y ago
  Sure but in reality most if not all educational institutes that I'm aware of have Matlab licenses, it's what everyone in that particular field uses, and it's better to publish something with Matlab code than nothing at all which is I guess the alternative (it's a means to an end after all). I can imagine this will change in the long run, but right now there are many valid reasons why people use these tools.
missosoup 7y ago
> There are entire ecosystems of academic libraries built around Matlab that can’t all just be picked up and moved to Python. They can and they are. Python is increasingly displacing everything in the data industry and especially proprietary legacy platforms like Matlab. The number of things you can do in Matlab but not Python is converging on 0, while the inverse is not even worth trying to count. Major universities are abandoning Matlab, Labview, SPSS, Minitab etc for Python, which is basically the end for them all. The next wave of CS/SE/DS/ML graduates had no exposure to Matlab. It'll linger in electrical engineering for a few more years but will suffer the same fate. In the end, proprietary platforms have no chance against FOSS. > Example: my girlfriends department writes a world-leading MRI analysis library in Matlab Siemens is leading the MRI industry and the only place where they're still using Matlab is the legacy platforms that aren't yet listed for updates or aren't worth updating. The actual leading stuff is done with the same ML tools as the rest of the industry, mostly Tensorflow. Siemens and GE both also have programs to engage and eventually acquire 3rd party ML platforms not a single one of which has anything to do with Labview or Matlab outside of occasionally interfacing with legacy components. https://www.siemens-healthineers.com/en-au/digital-health-solutions/value-themes/artificial-intelligence-in-healthcare https://www.siemens-healthineers.com/en-au/digital-health-so...
- euler_angles 7y ago
  > Major universities are abandoning Matlab, Labview, SPSS, Minitab etc for Python Just to add another point of anecdata. I helm a large data science effort in the defense industry. We are actively moving away from MATLAB and to Python. It's easier for us to find Python coders, easier to train people to use Python, more maintainable for the restrictions we have on our networks, and cheaper.
stoops 7y ago
In python, your MRI analysis library could be trivially hooked up with other cloud data pipelines. Companies would require fewer training courses on average. python is the most popular programming language in the world and getting better.

kensai 7y ago

Than I guess Julia will be an even more powerful language to learn, as it combines the flexibility of Python with all the use cases of all major technical/scientific languages, while being efficient and fast. https://juliacomputing.com/domains/ https://juliacomputing.com/domains/

bayesian_horse 7y ago
I think Julia is better designed than R. But it lacks R's ecosystem, so Julia is a tougher sell. I haven't done a lot with Julia, so I don't know if it's easier to teach Julia to novices than Python or R.
- elcritch 7y ago
  Julia’s ecosystem has been progressing since 1.0. The GLM.jl lib has become much better as has the data frame package. It’s more consistent than Python‘s data-science ecosystem since it’s not tying disparate C code together. But having strong types (actual, not mypy) helps make code more consistent. Still Julia’s ecosystem seems to be building more from R’s more solid academic approach.
  - bayesian_horse 7y ago
    Yes, Julia has a lot of benefits in some regards. The community is great. But small. For a lot of situations, I'd be hesitant to invest in Julia, because I don't know if the community will stay that way or if it fades away.
    - ViralBShah 7y ago
      Out of curiosity, how would you know that the community is large enough, or committed enough? For example, while Julia has been in development for almost 10 years, a lot of the community has now been around for 5 years. There's about 2,500 Julia packages, with the ability to call C, Fortran, R, Python, Java, etc. All the key community stats based on downloads, website views, videos show a healthy growth every year. While in absolute numbers, we may be at 20% of R or Python communities, I am always curious to understand what people mean when they say the community is too small. What would be a signal that a particular community is big enough?

hyperbovine 7y ago

> R is objectively worse than Python for almost all data science tasks If you meant to type "machine learning" I'd probably agree, but R is much much better for small scale data exploration, visualization and modeling (i.e. 95% of DS) than Python. Pandas is an absolute horror show of an API compared to dplyr, and the best plotting libraries for Python are just copying features from R. Lack of a magrittr style infix operator, though seemingly minor, actually emerges as a real pain point once you become accustomed to using it. R is inferior to Python as a programming language, no doubt about it -- but most data scientists are not programmers. Which is the point of TFA.

missosoup 7y ago
> but most data scientists are not programmers This is the crux of the problem with R and why R is increasingly blacklisted at large orgs. It attracts non-programmers which may have been okay 5 years ago but is no longer acceptable. With the exception of some engineering powerhouses hiring pure research PhDs to write R code, the trend established over the last 2 years is that fewer and fewer employers are hiring data scientists that aren't programmers. There are too many candidates who know data science and can also do data engineering and even generalist SE tasks. Non-programmer data scientists are not competitive in the industry anymore except that small top-end research niche that doesn't exist in most orgs. Which brings us back to the fact that R was a successful niche language that allowed non-programmers to write models, but that's simply not enough anymore. Businesses want models that can be plugged into production pipelines, models that can scale without needing a dedicated team to re-implement them, and they want staff who do engineering in addition to whatever it is they specialise in. Virtually all data scientists graduating today are programmers, and pretty good ones. Candidates who only know R can't compete against them. > Lack of a magrittr style infix operator, though seemingly minor, actually emerges as a real pain point once you become accustomed to using So you'd agree that you fall into the 'I prefer the syntax' bucket then? I don't really see any arguments against Python in your comment. Funnily enough, it's trivial to implement a pipe style operator in Python and there's at least two popular libraries for that.
- fock 7y ago

hatmatrix 7y ago

Whenever someone makes a statement that one language is "objectively" better than another, it is followed only by opinions of the author.

FranzFerdiNaN 7y ago

Statisticians, economists, biologists, social scientists all learn and work with R. They publish new packages in R, not in Python. There is no trend at all that this is moving towards Python. Python is far, very far behind when it comes to state of the art research in anything stat-related (besides machine learning i guess, but R is pushing hard to close that gap).

missosoup 7y ago
> Statisticians, economists, biologists, social scientists all learn and work with R They used to work with R. And the old generation of engineers used to work with Matlab. The old generation still does. The new generation has been using actual programming languages, typically Python, since high school. They were the first wave of graduates in 2019 that specialised in a discipline and were also competent in software engineering. The old generation is going to be driven out of the job market by the new in the span of 5 years as they saturate the senior tier of their respective fields. How do you compete for a job when all you know is R and your discipline, against someone who's a full fledged software engineer who knows your discipline and can put models directly into production use?
- bart_spoon 7y ago
  This simply isn't true. There are more of them that know Python these days in addition to R, but as a recent graduate of a respected statistics graduate program, I can assure you that R is still the overwhelmingly preferred choice in the field, and also is in economics.
- jointpdf 7y ago
  So assuming this is true, what plan of action do you recommend for an “old gen” data scientist who is strong in math, stats, ML theory, R, dataviz, ETL, research, etc., but who is not by a long shot a “full fledged software engineer”? I will soon be competing against this new crop of statistician/engineer superhybrids you speak of. I know a fair bit of Python (mostly for ML/DL applications), bash, and just a smidge of HTML/CSS/JS (just enough tweak a front end demo via R Shiny). I’m OCD enough that I make every effort to write clean and reproducible code and unit test it as I go (is this TDD?). I can implement some stats algorithms (e.g. EM algorithm, MCMC) from scratch with a pseudocode reference, but I rarely if ever have the occasion to do that for obvious reasons. I understand the concept of computational complexity, though I don’t have any figures memorized. But I’ve never taken any CS course beyond Programming 101. I wouldn’t know how to navigate a complex production codebase. Embarrassingly, I know almost nothing about git. I’m 100% sure I’d get slaughtered in a whiteboard interview or similar. For that matter, I could easily get nailed on some holes in my core data science knowledge (cough SQL cough). So, do I rebuild my foundation around software engineering, or just patch up the obvious holes? Grind towards a management position and let my inferior skills rot away?

bart_spoon 7y ago

I would say that for data manipulation and data visualization, R is objectively superior to Python. And for most statistical methods. Its pretty even for many machine learning algorithms. Python only really outclasses R in deep learning in my opinion.

deleted 7y ago

[deleted]