5 ms·
Whoa - [citation needed] on "Most of science has been debunked" We've learned to be much more skeptical, and the previous standards were scandalous, but there'
by orzig 6y ago
Whoa - [citation needed] on "Most of science has been debunked"
We've learned to be much more skeptical, and the previous standards were scandalous, but there's a difference between "too much of" and "most of"
- ndiscussion 6y agoHere's a start, almost every single field is implicated: https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis Economics, social sciences, even medicine.
- Der_Einzige 6y agoAnd don't forget machine learning! If your paper in ML doesn't provide code, I assume that it's not reproducible and the authors made a conscious decision to not release the code...
- QuesnayJr 6y agoSocial psychology has a replication crisis, because they had a culture of never questioning a published result. You can see that the vast majority of that article is devoted to problems within psychology. (Medicine also seems to have problems.) In economics 1/3 of 18 studies failed to replicate? A study that small, what are the odds that study itself would replicate? Note that this claim is restricted to experimental studies, which are usually testing human decision-making on economic questions. This is the field of economics closest to social psychology. It seems plausible to me that the model of "get a bunch of people in a room and ask them some questions" is not a reliable methodology. Economics as a whole may have a problem with replication, but we can't tell from the information on that page. There are whole fields that aren't even mentioned -- no physics, no chemistry.
- chalst 6y agoThat is not most of science. There are more biochemists than all those fields combined. The fields affected have three properties: they deal with complex entities, they are interested in phenomena that is generally investigated using statistical methods and they deal with noisy data. For those fields, we need new methodologies, but other sciences do not have a comparable problem.
- nradov 6y agoIoannidis JPA (2005) Why Most Published Research Findings Are False. PLoS Med 2(8): e124. doi:10.1371/journal.pmed.0020124 https://doi.org/10.1371/journal.pmed.0020124 https://doi.org/10.1371/journal.pmed.0020124
- berekuk 6y agoOn the other hand, here's a hilarious response how Ioannidis is wrong in his claim from one of the most well-known replication crisis blogs: https://replicationindex.com/2020/12/24/ioannidis-is-wrong-most-of-the-time/ https://replicationindex.com/2020/12/24/ioannidis-is-wrong-m...
- nradov 6y agoWhy do you suppose they chose to publish this hit piece as a blog post instead of going through the peer review process with a real journal article? Or at least a formal letter to PLoS Medicine?
- berekuk 6y agoI'm not sure. Maybe they did both and the real article is somethere in peer review pipeline. (Do you imply that no posts on replicationindex should be trusted since they're not properly peer reviewed?) One possible reason is that blog posts are more widely shared on social media, and I agree that this is kind of a hit piece, mostly as a reaction to Ioannidis' contrarian stance on COVID-19, so publicity was probably an objective here.