3 ms·
Oh absolutely! I totally understand you're not saying that's what's happening here, but just to clarify for other readers, I'd say in this case there are a few
by asgraham 8y ago
Oh absolutely!
I totally understand you're not saying that's what's happening here, but just to clarify for other readers, I'd say in this case there are a few reasons we don't need quite that level of suspicion:
1. There is a seemingly-sound theoretical basis for the observation (moreover, there doesn't seem to have been any element of fishing. Sometimes researchers think fishing is OK if they just do it once to avoid multiple testing. But then you certainly have multiple testing at the meta level).
2. This study agrees with at least one other completely independent, reputable study [1] (even different methodologies, apparently) that supported the underlying theory.
I do think the problem of multiple testing at the meta level is vastly underrated by the scientific community. Just, in this case, I think there are reasons to reject that hypothesis.
[1] https://scholar.princeton.edu/sites/default/files/lwantche/files/beninwkn-qje-final.pdf https://scholar.princeton.edu/sites/default/files/lwantche/f...