3 ms·
Pre-registration is nice, larger samples/greater power is necessary, and increasing the p-threshold may indirectly filter out some false positives but kind of m
by ramblenode 10y ago
Pre-registration is nice, larger samples/greater power is necessary, and increasing the p-threshold may indirectly filter out some false positives but kind of misses the underlying issue of p-hacking, some of which would be solved by pre-registration.
The authors suggestions are preventative in nature but what I would like to see above all else is requiring researchers to publish the raw data and to make their statistical analyses minimally reproducible--something which could be satisfied by publishing scripts or Excel macros along with instructions for any non-automated data stitching. Experiments frequently implode at the analysis phase which then gets intentionally or unintentionally masked in ambiguous, poorly written methods sections. Giving others access to the data allows errors to be spotted earlier after publication and alternative hypotheses and analyses to be tested against the published results. It's also sometimes the only way of spotting abnormalities resulting from the data collection process itself. Again, not a means of preventing errors, but a low friction way of discovering them. Maybe having everything in the open would light a fire under some researchers to be more thorough, though.