4 ms·

I don’t think the author has the causality correct here, at least for biosciences. The statistical problems existed long before omics, big data, etc. Most of

I don’t think the author has the causality correct here, at least for biosciences. The statistical problems existed long before omics, big data, etc. Most of the graduate programs don’t require students to take statistics, or if they do, it’s very cursory. Furthermore, students often learn very little about assay design - they end up thinking that non-linear responses are linear and do things like divide assay signals to get ratios (two sins here: assuming the assay response intercepts at 0 and that it’s linear). So at least for the biosciences, it’s been a shitshow for a while.

amelius 8y ago
One question is: would science come to a halt if people are not sufficiently careful, or would scientific discovery just slow down a bit? I'm asking because there will probably always be something you are not careful about. So in other words: how bad is bad statistics?
- Ericson2314 8y ago
  In a tree exploration, it's very possible to waste tons of time exploring dead end branches and never make it down the right one.
  - pixl97 8y ago
    There is likely more failure paths than available energy in the visible universe.
- jeffdavis 8y ago
  Why can't it go in reverse? New discoveries that are actually false seem likely to cause a lot of confusion leaving you worse than wrn you started.
  - iso1337 8y ago
    Indeed, you can look at this sirtuin boondoggle https://blogs.sciencemag.org/pipeline/archives/2011/09/22/the_latest_sirtuin_controversy https://blogs.sciencemag.org/pipeline/archives/2011/09/22/th... https://blogs.sciencemag.org/pipeline/archives/2011/12/13/the_sirtuin_saga https://blogs.sciencemag.org/pipeline/archives/2011/12/13/th... https://en.m.wikipedia.org/wiki/Sirtris_Pharmaceuticals https://en.m.wikipedia.org/wiki/Sirtris_Pharmaceuticals Famous professor from Harvard discovers possible route to elongating lifespan and then spends 1bn+ in funding to pursue that idea. I haven’t kept up with that story since then, so it’s possible that someone might be able to use it as a therapeutic target.
- iso1337 8y ago
  Part of the answer here comes down to the size of the effect. If the effect size is large, then you can be pretty sloppy with your statistics - in some ways it doesn’t matter because it’s almost a qualitative/binary difference. Obviously statistics becomes much more important when the effect size shrinks and you are squinting at some data trying to see if a 20% difference is real or not. Due to the poor foundations in biosciences (everything from the assays themselves, lack of assay replication, errors in interpretation) engineering/optimizing something can be fiendishly hard.
- olooney 8y ago
  Apocalyptic? Science would first grind to a halt, reverse itself, then descend into a thousand-year dark age of sophistry, apologetics, and superstition. Alexander Fleming and Jonas Salk would be laughingstocks, Vaccinations would be banned and plagues would visit upon the Earth. Norman Borlaug would be denounced, crop varieties would be chosen by astrology, and famine would be visit upon the Earth. The guilty party for any crime would be determined by Phrenology or Theranos blood tests, the punishment decided by Jungian psychoanalysis, and injustice would rule the Earth. I only say this because this is more or less what happened during the Mao's Chinese Cultural Revolution, Pol Pot's Khmer Rouge, or the European Dark ages. Historically speaking, it's a very bad sign when science is outright rejected and it most likely means millions of people are about to be murdered, and then millions more are going to starve to death or die of preventable diseases.

amelius 8y ago

One question is: would science come to a halt if people are not sufficiently careful, or would scientific discovery just slow down a bit? I'm asking because there will probably always be something you are not careful about. So in other words: how bad is bad statistics?

Ericson2314 8y ago
In a tree exploration, it's very possible to waste tons of time exploring dead end branches and never make it down the right one.
- pixl97 8y ago
  There is likely more failure paths than available energy in the visible universe.
jeffdavis 8y ago
Why can't it go in reverse? New discoveries that are actually false seem likely to cause a lot of confusion leaving you worse than wrn you started.
- iso1337 8y ago
  Indeed, you can look at this sirtuin boondoggle https://blogs.sciencemag.org/pipeline/archives/2011/09/22/the_latest_sirtuin_controversy https://blogs.sciencemag.org/pipeline/archives/2011/09/22/th... https://blogs.sciencemag.org/pipeline/archives/2011/12/13/the_sirtuin_saga https://blogs.sciencemag.org/pipeline/archives/2011/12/13/th... https://en.m.wikipedia.org/wiki/Sirtris_Pharmaceuticals https://en.m.wikipedia.org/wiki/Sirtris_Pharmaceuticals Famous professor from Harvard discovers possible route to elongating lifespan and then spends 1bn+ in funding to pursue that idea. I haven’t kept up with that story since then, so it’s possible that someone might be able to use it as a therapeutic target.
iso1337 8y ago
Part of the answer here comes down to the size of the effect. If the effect size is large, then you can be pretty sloppy with your statistics - in some ways it doesn’t matter because it’s almost a qualitative/binary difference. Obviously statistics becomes much more important when the effect size shrinks and you are squinting at some data trying to see if a 20% difference is real or not. Due to the poor foundations in biosciences (everything from the assays themselves, lack of assay replication, errors in interpretation) engineering/optimizing something can be fiendishly hard.
olooney 8y ago
Apocalyptic? Science would first grind to a halt, reverse itself, then descend into a thousand-year dark age of sophistry, apologetics, and superstition. Alexander Fleming and Jonas Salk would be laughingstocks, Vaccinations would be banned and plagues would visit upon the Earth. Norman Borlaug would be denounced, crop varieties would be chosen by astrology, and famine would be visit upon the Earth. The guilty party for any crime would be determined by Phrenology or Theranos blood tests, the punishment decided by Jungian psychoanalysis, and injustice would rule the Earth. I only say this because this is more or less what happened during the Mao's Chinese Cultural Revolution, Pol Pot's Khmer Rouge, or the European Dark ages. Historically speaking, it's a very bad sign when science is outright rejected and it most likely means millions of people are about to be murdered, and then millions more are going to starve to death or die of preventable diseases.