p=0.05 is an absurdly low bar even with an extremely well-defined hypothesis. Once you open things to p-value hacking (i.e. trying a bunch of hypotheses until o

p=0.05 is an absurdly low bar even with an extremely well-defined hypothesis. Once you open things to p-value hacking (i.e. trying a bunch of hypotheses until one seems significant), p=0.05 is almost guaranteed for one of your hypotheses and is hence meaningless. The physics community is pretty good about dealing with both of these shortcomings. You often need a lower p-value of around p=0.003 (3 sigma) or lower to say something was significant, and a few more sigmas to claim an actual discovery. And in addition to this, you're expected to include a trials factor to correct for multiple hypotheses. It's not perfect, but it makes claims of statistical significance more meaningful.