> it just as likely happened by chance as any other theory and, as such, you should leave it that: it happened by chance. That's wrong. That's very wrong, and

> it just as likely happened by chance as any other theory and, as such, you should leave it that: it happened by chance. That's wrong. That's very wrong, and it reinforces the confusion already surrounding 'significance'. p=0.05 means that there was a 5% chance of obtaining results at least as extreme as measured if the null hypothesis was true (i.e. there was no difference to observe). p=.06 is a 6% chance, and p=.04 is a 4% chance. None of those are "as likely as not", which would be 50%, or p=0.5. Nor is p=.05 an inherently meaningful cutoff - it's an arbitrary threshold on a continuum of possibilities, and indeed 'significance' is often set at p=.01 instead. Overemphasis of a cutoff leads to p-hacking, and treating significance as binary promotes bad studies (p=.04 is not as meaningful as p=.00001). This article is too credulous to an unproven result, but saying that all results >.05 are "as likely by chance" or not worth discussing is intensely wrong. The original blog post responded quite legitimately by indicating an unexpected-but-unproven result, and suggesting further study.