6 ms·

In academic biology, statistics is routinely considered an afterthought. In the eyes of many labs/PIs, their results are already real. You just have to do tho

In academic biology, statistics is routinely considered an afterthought. In the eyes of many labs/PIs, their results are already real. You just have to do those damn statistical tests so the editor will get off your back. Most biologists don't understand statistics either. To the majority, the unpaired T-Test is the only test that is needed. Ever. Doesn't matter if you have one or two tails, paired or unpaired trials, normally distributed population or skewed. Most biologists don't take proper statistics classes and most just don't care. I doubt most biologists would even be able to name alternative statistical tests.

markkat 15y ago
So very true. I went from physics into biology, and the shoddy application of statistics is surprising. This is a failure that could be easily addressed at the university level. More stats classes.
- onemoreact 15y ago
  The best and simplest approach IMO would be improving peer review. If it was understood that scientific journals would reject papers with poor statistical analysis you would see changes fairly rapidly.
  - markkat 15y ago
    Yes, but I think one of the problems is that the reviewers can't sufficiently judge statistics. It is peer-review, after all. I've had some reviewers ask some questions that indicated that they didn't really understand the t-test.
  - carbocation 15y ago
    Nature and the NEJM, and possibly others, assign statistical reviewers. So this is a known problem that some high profile journals are attempting to correct.
- bmuon 15y ago
  I think it doesn't have to be just "more stats classes". They also need to be focused on the kind of work they'll need to do with those statistics. If you teach future biologists or medical doctors "so there is this theorem that says that if there is statistical significance of A and not of B, then the value of P equals the..." they won't learn it/use it.
- jessriedel 15y ago
  Frankly, I was pretty blown away by how poorly it's understood by physicists (although they are probably better than biologist). Even in experimental particle physics, where statistics are essential, the statistical techniques are all self-taught and seat-of-the-pants. I am confident everything being released by the major particle physics experiments is statistically sound after it goes through the collaboration review process, but there are certainly poor inferences being made at the level of the individual physicist. I listened to at least a couple of confused conversations between smart particle physicists about whether frequentism or Bayesianism was "right".
gaurav_v 15y ago
The prevailing attitude in my lab is that "if you need statistics, you did the wrong experiment."
- markkat 15y ago
  What? How would that be justified?
  - polyfractal 15y ago
    I've seen this attitude before. The idea goes something like: "If the effect you are seeing is so subtle that you have to resort to statistical tests, it probably isn't a phenotype you should be studying". In my experience, this usually comes from PIs that worked back in the days of "move bar of light across visual field, see neurons fire". Their experiments were ground breaking and had drastic phenotypes. On the flipside, I've seen experiments where they are showing that one population of neurons is 2% more dynamic than another population, and sometimes you just wonder if that really matters at all (or if you are really sampling the population well enough)
  - ArbitraryLimits 15y ago
    Actually, that quotation is usually attributed to Ernest Rutherford. This is a common discussion among particle physicists, but of course Rutherford did his work before the statistical interpretation of quantum mechanics became clear. For example, even measuring something like the mass of a particle is inherently probabilistic; for short-lived particles it's usually done by repeating an experiment many times, fitting a gaussian distribution to the decay times and backing the mass out from the time-energy uncertainty principle. See http://www-stat.stanford.edu/~ckirby/brad/papers/2005NEWModernScience.pdf http://www-stat.stanford.edu/~ckirby/brad/papers/2005NEWMode... for a really interesting discussion of the way things are headed from a statistician's perspective.
  - jessriedel 15y ago
    To elaborate on what other have said: it's not meant to be taken literally. There will of course be cases (e.g. the LHC) where it's imperative to do precise statistics because it's so expensive to make a measurement and only a few measurements can be reasonably made. But the idea of the quote is that often a researcher is going to be wasting his time if the effect he is looking at is so small as to require heavy duty statistics; the fancy statistical machinery can be a time-sink and distraction from the fact that the underlying effect isn't that interesting.
ugh 15y ago
Social science has exactly the same problem. More statistic classes are desperately needed.
- gospelwut 15y ago
  I'm not a hardcore statistics person, since I only took enough to get me through ECE, but I think this is why I always had a problem with softer sciences (undergraduates, maybe graduate students are better). Things always boiled down to, "look, it's obvious." My background simply makes me think the obvious isn't always right.
  - ugh 15y ago
    Funnily enough that’s actually one of the reasons why I have always been fascinated by social science. Not as something where the results and methods are fascinating but rather as something where much is to be done and all problems are hard to solve. During the last century empiricism’s role in the social science has been growing but social science as an empirical science is still relatively young and, I think more importantly, it’s not really dominated by empiricism the same way, say, physics is. It’s still necessary to adapt and find methods.
    - gospelwut 15y ago
      I think, in general, I find some of academic's dogma a bit off-putting. I always approach things in the most classical sense--that is, everything is relatively true. Not to dismiss empirical data though. For example, people seem entrenched in saying dark matter is X or will do Y, but even people who helped discover its possibility say to be cautious of making such statements. When people come out and say it's not definitive in its existence (yet), they tend to get lambasted. I know there is always politics and evangelizing in every human situation, but my childhood notions of a pure method/approach seem a bit jaded now. Perhaps, some of the softer sciences need to study inter-academic hard science ;)
- achompas 15y ago
  In my experience, economics is the exception. Many of the economists I've worked with know their statistics, and they put in a lot of work to make sure their models agree with econometric or time series theory (that's possibly selection bias, though, given where I've worked). Also, apologies for perpetuating the "arrogant economist" meme. Just reporting what I've experienced. :P

markkat 15y ago

So very true. I went from physics into biology, and the shoddy application of statistics is surprising. This is a failure that could be easily addressed at the university level. More stats classes.

onemoreact 15y ago
The best and simplest approach IMO would be improving peer review. If it was understood that scientific journals would reject papers with poor statistical analysis you would see changes fairly rapidly.
- markkat 15y ago
  Yes, but I think one of the problems is that the reviewers can't sufficiently judge statistics. It is peer-review, after all. I've had some reviewers ask some questions that indicated that they didn't really understand the t-test.
- carbocation 15y ago
  Nature and the NEJM, and possibly others, assign statistical reviewers. So this is a known problem that some high profile journals are attempting to correct.
bmuon 15y ago
I think it doesn't have to be just "more stats classes". They also need to be focused on the kind of work they'll need to do with those statistics. If you teach future biologists or medical doctors "so there is this theorem that says that if there is statistical significance of A and not of B, then the value of P equals the..." they won't learn it/use it.
jessriedel 15y ago
Frankly, I was pretty blown away by how poorly it's understood by physicists (although they are probably better than biologist). Even in experimental particle physics, where statistics are essential, the statistical techniques are all self-taught and seat-of-the-pants. I am confident everything being released by the major particle physics experiments is statistically sound after it goes through the collaboration review process, but there are certainly poor inferences being made at the level of the individual physicist. I listened to at least a couple of confused conversations between smart particle physicists about whether frequentism or Bayesianism was "right".

gaurav_v 15y ago

The prevailing attitude in my lab is that "if you need statistics, you did the wrong experiment."

markkat 15y ago
What? How would that be justified?
- polyfractal 15y ago
  I've seen this attitude before. The idea goes something like: "If the effect you are seeing is so subtle that you have to resort to statistical tests, it probably isn't a phenotype you should be studying". In my experience, this usually comes from PIs that worked back in the days of "move bar of light across visual field, see neurons fire". Their experiments were ground breaking and had drastic phenotypes. On the flipside, I've seen experiments where they are showing that one population of neurons is 2% more dynamic than another population, and sometimes you just wonder if that really matters at all (or if you are really sampling the population well enough)
- ArbitraryLimits 15y ago
  Actually, that quotation is usually attributed to Ernest Rutherford. This is a common discussion among particle physicists, but of course Rutherford did his work before the statistical interpretation of quantum mechanics became clear. For example, even measuring something like the mass of a particle is inherently probabilistic; for short-lived particles it's usually done by repeating an experiment many times, fitting a gaussian distribution to the decay times and backing the mass out from the time-energy uncertainty principle. See http://www-stat.stanford.edu/~ckirby/brad/papers/2005NEWModernScience.pdf http://www-stat.stanford.edu/~ckirby/brad/papers/2005NEWMode... for a really interesting discussion of the way things are headed from a statistician's perspective.
- jessriedel 15y ago
  To elaborate on what other have said: it's not meant to be taken literally. There will of course be cases (e.g. the LHC) where it's imperative to do precise statistics because it's so expensive to make a measurement and only a few measurements can be reasonably made. But the idea of the quote is that often a researcher is going to be wasting his time if the effect he is looking at is so small as to require heavy duty statistics; the fancy statistical machinery can be a time-sink and distraction from the fact that the underlying effect isn't that interesting.

ugh 15y ago

Social science has exactly the same problem. More statistic classes are desperately needed.

gospelwut 15y ago
I'm not a hardcore statistics person, since I only took enough to get me through ECE, but I think this is why I always had a problem with softer sciences (undergraduates, maybe graduate students are better). Things always boiled down to, "look, it's obvious." My background simply makes me think the obvious isn't always right.
- ugh 15y ago
  Funnily enough that’s actually one of the reasons why I have always been fascinated by social science. Not as something where the results and methods are fascinating but rather as something where much is to be done and all problems are hard to solve. During the last century empiricism’s role in the social science has been growing but social science as an empirical science is still relatively young and, I think more importantly, it’s not really dominated by empiricism the same way, say, physics is. It’s still necessary to adapt and find methods.
  - gospelwut 15y ago
    I think, in general, I find some of academic's dogma a bit off-putting. I always approach things in the most classical sense--that is, everything is relatively true. Not to dismiss empirical data though. For example, people seem entrenched in saying dark matter is X or will do Y, but even people who helped discover its possibility say to be cautious of making such statements. When people come out and say it's not definitive in its existence (yet), they tend to get lambasted. I know there is always politics and evangelizing in every human situation, but my childhood notions of a pure method/approach seem a bit jaded now. Perhaps, some of the softer sciences need to study inter-academic hard science ;)
achompas 15y ago
In my experience, economics is the exception. Many of the economists I've worked with know their statistics, and they put in a lot of work to make sure their models agree with econometric or time series theory (that's possibly selection bias, though, given where I've worked). Also, apologies for perpetuating the "arrogant economist" meme. Just reporting what I've experienced. :P