3 ms·
I think your take-away from the reproducibility crisis shouldn't be that science is corrupt, peer review is stuffed with cronies, or scientists don't know what
by nostrademons 9y ago
I think your take-away from the reproducibility crisis shouldn't be that science is corrupt, peer review is stuffed with cronies, or scientists don't know what they're doing, it should be that science is really fucking hard:
https://fivethirtyeight.com/features/science-isnt-broken/#part1 https://fivethirtyeight.com/features/science-isnt-broken/#pa...
My old boss at Google used to run the data science program for Search - this is basic business intelligence, not even the natural sciences. He gave an hour and a half long presentation once on all the ways our experiments can and have gone wrong. It included everything from browser bugs disrupting logging, to service outages that happened in the middle of the experiment, to current events that change the normal traffic pattern, to software bugs that trigger failsafes that shut off the experiment but don't affect the control. By the end of the presentation, engineers were asking "So how is it that we actually get anything right?"
My brother-in-law is an actual scientist - he does cancer research, studying gene expression within head & neck tumors. The time it takes to get one rigorous, publishable result is measured in years. And it can be derailed by months by something as simple as a tech mixing up lab samples.
The thing is, when people use examples of these techniques going wrong as examples of why science is untrustworthy, they forget that normal human reasoning doesn't even bother trying. It's basically a given that most of the stuff you "know" is false. What would be the reproducibility stats for an everyday human belief like "vaccines cause autism" or "the economy does better under Democratic presidents" or "vitamin C prevents colds"?
The virtue of science is in being less wrong, it's not in being right. Which, given how often the average human is wrong, is something I'll happily take.
- babyrainbow 9y ago> everyday human belief like "vaccines cause autism".. A bit funny that you have to tell this, considering where this originated from..