3 ms·
The most salient detail about the way methodological errors compound is that by engaging in more replications, you can beat down the rate of spurious results in
by p4wnc6 11y ago
The most salient detail about the way methodological errors compound is that by engaging in more replications, you can beat down the rate of spurious results in the replications. It's of little importance that it is multiplicative with the flaw rate of the original, because the flaw rate of a set of independent replications will go to zero super fast in the size of the replications.
If the flaw rate is >= 50% and the number of replications is low, it presents a case where it's of little social value to do replications, depending on their cost (which is usually high).
From your original comment, the way a naive reader will view it is that replications often don't work, because you primed them with a number you just pulled out of nowhere (50%). That will matter much more than the other detail about it being multiplicative, which doesn't really matter much at all no matter what the flaw rates are, because it all hinges on the flaw rates of the replications and how many replications you do anyway.
I know that you did not mean to misrepresent it with the 50% number. I'm just harping on this because by happening to use a number like 50% it has the potential to distract and do more harm than good, regardless of anything else in your comment or even whether or not the 50% had anything to do with your main point.