4 ms·
A lot of this is simply that p < 0.05 is a very low bar for research. If we want to understand how the world works, we need to be able to construct mechanisms t
by sarbaz 8y ago
A lot of this is simply that p < 0.05 is a very low bar for research. If we want to understand how the world works, we need to be able to construct mechanisms that explain why the results we measure in experiments come about. Without this thing, it's not science.
A good mechanism that accurately explains some part of the world can be validated with a much better result than p < 0.05, because it can be properly isolated from other factors and its effect size can be strengthened.
Without a mechanism to explain how things work, the experimental results that get published are probably just noise. And because p < 0.05 is such a low bar to pass, lots of noise gets published. Trying to reproduce an experiment whose results were drawn from /dev/urandom is pointless.
In fact I'm surprised the replication was so high. I was expecting it to be around 5% (i.e. results are totally random). So maybe there is some hidden merit to all of this
- nicoburns 8y ago> If we want to understand how the world works, we need to be able to construct mechanisms that explain why the results we measure in experiments come about. This, 1000 times over. Unfortunately we seem to teach that statistical analysis is sufficient to infer causation in a lot our undergraduate science programs.
- traverseda 8y agoI wouldn't expect totally random because people have common sense. I'd expect it to be similar to the folk-wisdom of the time, since a lot of it is going to be repackaging of that same folk-wisdom, and negative results tend not to get published, or get massaged until they're positive. I'd expect social-psychology to mirror the dominant opinions of people in other segments of the humanities, although it is of course hard to determine the direction of that relationship.