The issue is that you probably wouldn't do two tests if you'd planned a short one. So you'd randomly get an overly strong, or overly weak, result. The actual e

The issue is that you probably wouldn't do two tests if you'd planned a short one. So you'd randomly get an overly strong, or overly weak, result. The actual effect depends on your data and how noisy it is, and if that correlates to external factors, etc. You need to know the characteristics of your data then you can decide on your desired confidence and only then pick a test designed to give you that based on your data.