It is somewhat arbitrary. 0.05 corresponds to roughly 2, and 0.01 to roughly 3 standard deviations under the normal distribution. In high energy physics, for

It is somewhat arbitrary. 0.05 corresponds to roughly 2, and 0.01 to roughly 3 standard deviations under the normal distribution. In high energy physics, for example, in the past a result with 3 sigma significance was considered “a discovery”, but it turned out there were a lot of false signals, so it was upped to 5 sigma (~ 3 x 10^{-7} percent chance of getting the effect assuming there is no effect in reality), while 3 sigma (or 4) result is considered “evidence of effect“. Unfortunately in softer sciences, considering their relatively generally poor state of research standards, we have no idea if 0.05 or 0.01 levels really are good standards of “discovery”.