3 ms·
Why are people so obsessed with publication bias and statistical power? Those are just two of the dozens of reasons why published research is unreliable, and it
by Alex3917 11y ago
Why are people so obsessed with publication bias and statistical power? Those are just two of the dozens of reasons why published research is unreliable, and it's not even clear that they're the most important ones.
It seems like the skeptic community just randomly glommed onto those issues in the 90s or something and haven't updated their worldview since.
- 467568985476 11y agoWhat are the other, more important reasons that research is unreliable?
- Alex3917 11y agoI have a list of them going here, but it's definitely not comprehensive either: http://www.alexkrupp.com/Citevault.html#pharmaceuticals http://www.alexkrupp.com/Citevault.html#pharmaceuticals Even if you look just at statistical analysis, there are dozens of different issues. E.g. I seem to remember some articles about how basically 100% of research involving brain scans is incorrect, because of a mix of statistical problems and also issues related to the imaging software.
- tcj_phx 11y agoThis section is particularly useful to me: http://www.alexkrupp.com/Citevault.html#mentalhealth http://www.alexkrupp.com/Citevault.html#mentalhealth Thanks!
- aschreyer 11y agoA big problem are the perverse incentives researchers are facing. The only thing that matters is publishing in prestigious journals, otherwise your career is pretty much over (no real chance to get tenure). Universities often don't even check the publications, and researchers usually omit the actual title of the publication and only keep the name of the journal. Moreover, if you want to get tenure you need to work on "hot" topics (as in CRISPR at the moment or cancer in general), meaning you don't necessarily spend too much time on one subject. I'm sure researchers are often aware that the research results are not ready for publication yet, but it's either publish or perish, and you usually get away with publishing sub-par results because no one really follows up on them either. Cancer and Alzheimer are two fields were billions have been wasted by research grants and pharma companies because the initially promising drug targets for example could not really be validated. I'm afraid this will stay that way as long as the career dilemma researchers are facing does not change.
- jessaustin 11y agoI suppose I haven't read enough about publication bias to get as annoyed by it as you seem. TFA offers a pretty convincing argument that publication bias is bad enough to doom most current scientific efforts to irrelevance. That sounds like something we should talk about? Or we can talk about more important reasons why published research is unreliable, if you care to mention them. What is "the skeptic community"? Is Reason part of it? I'm a bit skeptical that's even a thing... Statistical power is mentioned once in TFA and then forgotten. Amusingly enough, it gets the relevance of statistical power for the subject under discussion backwards. That graf might actually have been added by an "obsessed" editor.
- orblivion 11y agoReason is generally a libertarian publication and website. For the Skeptic Community go to r/skeptic. They're related to the Atheist Community.
- akiselev 11y agoSo by "skeptic community" you actually mean "reddit skeptic community?"
- deleted 11y ago[deleted]
- Alex3917 11y agoThere are skeptic conferences, skeptic magazines, skeptic books, skeptic authors, skeptic web forums, skeptic TV shows, etc. I think this easily meets pretty much any definition of community.
- jessaustin 11y agoThis seems to be focused toward critiques of religion rather than of science? I don't see the connection.