4 ms·

Where is the discussion of R? You talk of scientific computation and don't speak of R? That's an oversight, given the majority of the scientists I know have use

Where is the discussion of R? You talk of scientific computation and don't speak of R? That's an oversight, given the majority of the scientists I know have used R. There's also STATA, which economists love, which can do somethings in my workflow much quicker than R. There is also a huge contingent of analysts that uses SAS, especially in healthcare and finance. The car analogies in the blogpost are not particularly useful... Why do people feel the need to dumb down a topic with off-the-wall analogies? Talking about Julia like it's Tesla is laughable. Tesla is a huge innovator, Julia is another tool that does similar things to the other tools. The apt analogy for Julia would be a new ICE company, not a new EV company.

BeetleB 7y ago
Yes, but the majority of computational scientists do not use statistics - something that may be hard to believe. As an example, most physics programs don't even have an introductory statistics class in their curriculum. Some engineering disciplines use it more, and it is often relied a lot in industry. But most engineering research makes little to no use of it.
- kgwgk 7y ago
  "If your experiment needs statistics, you ought to have done a better experiment." Lord Rutherford (maybe)
  - tylermw 7y ago
    Except there's a whole field of statistics called Design of Experiments, which describes the mathematical basis for how to create a good experiment :-)
    - kgwgk 7y ago
      In physics you often have the luxury, not always available in other disciplines, of having a good theory :-)
      - tylermw 7y ago
        Oh, I know (I have a PhD in physics)--but I'd say the main reason statistics isn't used in most fields of physics isn't because it wouldn't help, but rather because most physicists don't know much about statistics. And publication-wise, it's a bit of a chicken and the egg problem: since most reviewers are also not well-versed in statistics, they are less likely to accept a paper that uses statistical methods to confirm or reject a hypothesis. Thus, there's no institutional pressure for a physicist to learn statistics. (And a graduate course in statistical mechanics does not count, saying that as someone who used to use that excuse)
        BeetleB 7y ago
        >And publication-wise, it's a bit of a chicken and the egg problem: since most reviewers are also not well-versed in statistics, they are less likely to accept a paper that uses statistical methods to confirm or reject a hypothesis. Thus, there's no institutional pressure for a physicist to learn statistics. Not to mention that if experimentalists actually used statistics, they would have to describe their experiment in more detail when publishing for their statistical analysis to make sense. And in my experience, they really prefer to list as little about their experimental setup as they can get away with. >And a graduate course in statistical mechanics does not count, saying that as someone who used to use that excuse Heh heh. Most physicists I know would invoke "But we use probability in quantum mechanics all the time!"
        tylermw 7y ago
        > Most physicists I know would invoke "But we use probability in quantum mechanics all the time!" I have absolutely heard that one before :-)
        duguxu 7y ago
        I agree with you but IMO, this could be a good thing sometimes. Avoiding statistics to some extent requires more deterministic experiments, where statistics are limited to quantum uncertainty or gaussian distribution. Otherwise there is probably some unknown mechanism to be revealed, and that’s exactly what the community expects physicists to do.
  - gnufx 7y ago
    I think a variant from A Random Walk in Science was "The experiment failed, so we had to use statistics", which I appreciate up to a point. However, I'd hope that physicists calculate error estimates in some sense, and goodness of fit. You hear the statistical significance of results from CERN, for instance.
roel_v 7y ago
There is a lot more to scientific computation than statistics, and the only thing R is good at is statistics.
- kgwgk 7y ago
  It's quite popular in bioinformatics as well: https://www.bioconductor.org/ https://www.bioconductor.org/
  - lbeltrame 7y ago
    Unfortunately the quality of the Bioconductor packages varies wildly, from pretty good to downright abysmal. Some can't even work: take a look at this one[1], which is part of the current release of BioC: it will never work because the hardcoded host there is no longer functioning and the author wrote that for their master's thesis and have since moved on. In addition, there's no centralized bug reporting, and some packages use GH issues while for others you may need to email the author and hope for the best (sometimes works, sometimes doesn't). I use BioC regularly every day because I have no other alternatives, but its main advantage is the sheer numbers of packages available for almost every bioinformatic niche, rather than the quality of the packages themselves. [1] https://github.com/AllenTiTaiWang/anamiR/blob/7a7a133c553f8c3262cec65d86735fe121049456/R/database_support.R#L80 https://github.com/AllenTiTaiWang/anamiR/blob/7a7a133c553f8c...
- tylermw 7y ago
  That's not a true statement. R has packages for a wide variety of fields--statistics is just the most well developed. And an important aspect of scientific computation is data visualization, an area in which R is decidedly more advanced than other languages.
  - geoalchimista 7y ago
    One thing I can think of is that R is not well suited for computationally expensive work --- by "computationally expensive" I mean something that takes months to run on a cluster. Not only is R 100--1000x slower than Fortran, but it also does not have strong support for parallelism or GPU computing. I have yet to see a computational fluid dynamicist using R. Though Python is also not much better in that aspect, it can be used as a glue language to interact with native C/C++/Fortran code under the hood.
ballenarosada 7y ago
"Julia is a huge innovation; a Tesla is another car that does similar things to the other cars."

BeetleB 7y ago

Yes, but the majority of computational scientists do not use statistics - something that may be hard to believe. As an example, most physics programs don't even have an introductory statistics class in their curriculum. Some engineering disciplines use it more, and it is often relied a lot in industry. But most engineering research makes little to no use of it.

kgwgk 7y ago
"If your experiment needs statistics, you ought to have done a better experiment." Lord Rutherford (maybe)
- tylermw 7y ago
  Except there's a whole field of statistics called Design of Experiments, which describes the mathematical basis for how to create a good experiment :-)
  - kgwgk 7y ago
    In physics you often have the luxury, not always available in other disciplines, of having a good theory :-)
    - tylermw 7y ago
      Oh, I know (I have a PhD in physics)--but I'd say the main reason statistics isn't used in most fields of physics isn't because it wouldn't help, but rather because most physicists don't know much about statistics. And publication-wise, it's a bit of a chicken and the egg problem: since most reviewers are also not well-versed in statistics, they are less likely to accept a paper that uses statistical methods to confirm or reject a hypothesis. Thus, there's no institutional pressure for a physicist to learn statistics. (And a graduate course in statistical mechanics does not count, saying that as someone who used to use that excuse)
      - BeetleB 7y ago
        >And publication-wise, it's a bit of a chicken and the egg problem: since most reviewers are also not well-versed in statistics, they are less likely to accept a paper that uses statistical methods to confirm or reject a hypothesis. Thus, there's no institutional pressure for a physicist to learn statistics. Not to mention that if experimentalists actually used statistics, they would have to describe their experiment in more detail when publishing for their statistical analysis to make sense. And in my experience, they really prefer to list as little about their experimental setup as they can get away with. >And a graduate course in statistical mechanics does not count, saying that as someone who used to use that excuse Heh heh. Most physicists I know would invoke "But we use probability in quantum mechanics all the time!"
        tylermw 7y ago
        > Most physicists I know would invoke "But we use probability in quantum mechanics all the time!" I have absolutely heard that one before :-)
      - duguxu 7y ago
        I agree with you but IMO, this could be a good thing sometimes. Avoiding statistics to some extent requires more deterministic experiments, where statistics are limited to quantum uncertainty or gaussian distribution. Otherwise there is probably some unknown mechanism to be revealed, and that’s exactly what the community expects physicists to do.
- gnufx 7y ago
  I think a variant from A Random Walk in Science was "The experiment failed, so we had to use statistics", which I appreciate up to a point. However, I'd hope that physicists calculate error estimates in some sense, and goodness of fit. You hear the statistical significance of results from CERN, for instance.

roel_v 7y ago

There is a lot more to scientific computation than statistics, and the only thing R is good at is statistics.

kgwgk 7y ago
It's quite popular in bioinformatics as well: https://www.bioconductor.org/ https://www.bioconductor.org/
- lbeltrame 7y ago
  Unfortunately the quality of the Bioconductor packages varies wildly, from pretty good to downright abysmal. Some can't even work: take a look at this one[1], which is part of the current release of BioC: it will never work because the hardcoded host there is no longer functioning and the author wrote that for their master's thesis and have since moved on. In addition, there's no centralized bug reporting, and some packages use GH issues while for others you may need to email the author and hope for the best (sometimes works, sometimes doesn't). I use BioC regularly every day because I have no other alternatives, but its main advantage is the sheer numbers of packages available for almost every bioinformatic niche, rather than the quality of the packages themselves. [1] https://github.com/AllenTiTaiWang/anamiR/blob/7a7a133c553f8c3262cec65d86735fe121049456/R/database_support.R#L80 https://github.com/AllenTiTaiWang/anamiR/blob/7a7a133c553f8c...
tylermw 7y ago
That's not a true statement. R has packages for a wide variety of fields--statistics is just the most well developed. And an important aspect of scientific computation is data visualization, an area in which R is decidedly more advanced than other languages.
- geoalchimista 7y ago
  One thing I can think of is that R is not well suited for computationally expensive work --- by "computationally expensive" I mean something that takes months to run on a cluster. Not only is R 100--1000x slower than Fortran, but it also does not have strong support for parallelism or GPU computing. I have yet to see a computational fluid dynamicist using R. Though Python is also not much better in that aspect, it can be used as a glue language to interact with native C/C++/Fortran code under the hood.

ballenarosada 7y ago

"Julia is a huge innovation; a Tesla is another car that does similar things to the other cars."