2 ms·
There's a huge problem in many branches of science where, using normal methodology, it's possible to come up with an endless stream of results that are unlikely
by pjscott 4y ago
There's a huge problem in many branches of science where, using normal methodology, it's possible to come up with an endless stream of results that are unlikely to have any predictive validity outside of the extremely specific circumstances of the experiment -- and often not even there. This gives the illusion of knowledge, but everything you "know" falls apart if you actually try to apply it to the world around you.
https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis
An attempt by DARPA to evaluate the various fields of social science found that only about half of the papers they tried to replicate actually gave the same results when tried a second time. People in those fields could give pretty good predictions of which papers would replicate, but the publishing apparatus doesn't seem to have much in the way of quality control to filter out the papers everyone knows are junk -- nor do people cite the junk studies less often. Here's a very readable writeup with details:
https://fantasticanachronism.com/2020/09/11/whats-wrong-with-social-science-and-how-to-fix-it/ https://fantasticanachronism.com/2020/09/11/whats-wrong-with...
What's more, correlational studies have another problem: even a consistent-but-small correlation between two variables usually is non-causal when you're looking at sufficiently complicated webs of cause and effect, e.g. the ones found in the social sciences like psychology. The math here has some profound, dire implications, and I found this article to be a huge eye-opener:
https://www.gwern.net/Causality https://www.gwern.net/Causality