The thing you seem to be missing is that there's no one number that's a meaningful limit for all purposes. What probability you accept as significant should de

The thing you seem to be missing is that there's no one number that's a meaningful limit for all purposes. What probability you accept as significant should depend entirely on how you plan to use the results. Something with a p value of a staggering 70 % (i.e. it's more likely not true than true) is significant if the payoff is good when it's true, and the cost is small when it's not true. And 70 % is very far from 5 %! Then again, if the payoff is tiny compared to the cost, you might ask for a p-value of less than 0.01 %, in order for it to make sense to take the chance on it. Think like a poker player: a hand that has 1/4 chance of winning needs better than a 3-to-1 payout when it wins to be playable. Conversely, when the pot offers you a 3-to-1 payout, you better make sure your hand has more than a 1/4 chance of winning.