4 ms·

The impression that schools and media gives of how "science" is being implemented these days is inaccurate enough that I would call it propaganda. The reality i

by nonbel 10y ago

The impression that schools and media gives of how "science" is being implemented these days is inaccurate enough that I would call it propaganda. The reality is that the research literature is filled with over 90% junk, and a small percent of value, just like everything else. The funding and publication system is set up to force this junk percentage as high as the public will tolerate, so I do not think this "positive-spin" type of propaganda is a good thing.

deleted 10y ago
[deleted]
lordCarbonFiber 10y ago
I think your interpretation of the situation is a bit off. The 90% number, best I can tell, comes from a rather limited meta study which found the papers they tried to replicate had a low repoducablity rate. Note this doesn't immediately make them "junk" just adds another data point in support of the null hypothesis or in some cases points to the existence of confounding factors [0]. Looking at research literature at the single paper level is the realm of pulp science and media puff pieces. In terms of the larger field having a broad corpus to check your results against gives context to an experiment's result and makes it easier to weed out outliers. It's my opinion that a funding system that ends up with more noise than ideal is still a better state than just giving the reins to the pundits of the world who make conclusions without even pretending to have any experimental foundation. [0]https://www.washingtonpost.com/news/speaking-of-science/wp/2015/08/27/trouble-in-science-massive-effort-to-reproduce-100-experimental-results-succeeds-only-36-times/ https://www.washingtonpost.com/news/speaking-of-science/wp/2...
- nonbel 10y ago
  >"this doesn't immediately make them "junk" just adds another data point in support of the null hypothesis" Unless the null hypothesis happens to correspond to someone's actual hypothesis, chances are your study was designed to be junk from the beginning and anything of value that occurred was despite that. For example, no one actually believes humans have exactly zero impact on climate, or smoking has exactly zero impact on cancer incidence. Yet the vast majority of studies on these topics will be testing those "hypotheses". That is the junk.
Angostura 10y ago
> The reality is that the research literature is filled with over 90% jun Based on what? There is a problem of non-replicated papers - but where exactly are you getting your claim of 9 out of 10 - particularly in high quality journals?
- nonbel 10y ago
  Personal experience. It is great that the replication issue is getting attention, but really that is a minor aspect of the problem. Producing reproducible reports of your research is supposed to be the easy part... The real difficulty is how results are interpreted. What I was taught to do (along with basically my entire worldwide cohort of research colleagues) is test a "null hypothesis" rather than my hypothesis. Then when the null hypothesis was deemed to be inconsistent with the data, I was supposed to conclude my hypothesis was supported, with at most some hand-waving about other explanations. A crucial part of scientific research is distinguishing between different explanations for the results, and this is almost never found (at least for fields like medicine, psych, etc). If we can get the public eye to look behind that curtain, oh man...

by nonbel 10y ago

deleted 10y ago

[deleted]

lordCarbonFiber 10y ago

I think your interpretation of the situation is a bit off. The 90% number, best I can tell, comes from a rather limited meta study which found the papers they tried to replicate had a low repoducablity rate. Note this doesn't immediately make them "junk" just adds another data point in support of the null hypothesis or in some cases points to the existence of confounding factors [0]. Looking at research literature at the single paper level is the realm of pulp science and media puff pieces. In terms of the larger field having a broad corpus to check your results against gives context to an experiment's result and makes it easier to weed out outliers. It's my opinion that a funding system that ends up with more noise than ideal is still a better state than just giving the reins to the pundits of the world who make conclusions without even pretending to have any experimental foundation. [0]https://www.washingtonpost.com/news/speaking-of-science/wp/2015/08/27/trouble-in-science-massive-effort-to-reproduce-100-experimental-results-succeeds-only-36-times/ https://www.washingtonpost.com/news/speaking-of-science/wp/2...

nonbel 10y ago
>"this doesn't immediately make them "junk" just adds another data point in support of the null hypothesis" Unless the null hypothesis happens to correspond to someone's actual hypothesis, chances are your study was designed to be junk from the beginning and anything of value that occurred was despite that. For example, no one actually believes humans have exactly zero impact on climate, or smoking has exactly zero impact on cancer incidence. Yet the vast majority of studies on these topics will be testing those "hypotheses". That is the junk.

Angostura 10y ago

> The reality is that the research literature is filled with over 90% jun Based on what? There is a problem of non-replicated papers - but where exactly are you getting your claim of 9 out of 10 - particularly in high quality journals?

nonbel 10y ago
Personal experience. It is great that the replication issue is getting attention, but really that is a minor aspect of the problem. Producing reproducible reports of your research is supposed to be the easy part... The real difficulty is how results are interpreted. What I was taught to do (along with basically my entire worldwide cohort of research colleagues) is test a "null hypothesis" rather than my hypothesis. Then when the null hypothesis was deemed to be inconsistent with the data, I was supposed to conclude my hypothesis was supported, with at most some hand-waving about other explanations. A crucial part of scientific research is distinguishing between different explanations for the results, and this is almost never found (at least for fields like medicine, psych, etc). If we can get the public eye to look behind that curtain, oh man...