> This usually happens because someone runs a one-tailed test that ends up being overpowered. It always pains me a little when people doing research describe s

> This usually happens because someone runs a one-tailed test that ends up being overpowered. It always pains me a little when people doing research describe statistical power as a type of curse. Overpowered? Should we reduce it? The risk isn't having too much power, the risk is that someone will incorrectly interpret their Null Hypothesis Significance Test (NHST). They need to shift their focus to measuring something (and quantifying the uncertainty of their measurements), rather than think of "how likely was this result given a null hypothesis", whether that hypothesis is.. something is not greater than 0 (one-tail), or something is not 0 (two-tail). > You’ll often see statistical power conveyed as P90 or 90%. In other words, if there’s a 90% chance A is better than B, there’s a 10% chance B is better than A and you’ll actually get worse results. This isn't necessarily true. A could be the same as B. Also, these tests are being done from the frequentist perspective, so saying "there's X chance B is better than A" is inappropriate, unless you're talking about the conclusions of your significance test (e.g. 90% chance you correctly detect a difference between them--a difference you assume is fixed to some true underlying value). Overall, being aware that a one-tail test is taking the position that nothing can happen in the other direction is useful, but a good next step is understanding what NHST can and cannot say. This even a frequentist vs bayesian problem, since you could create situations where a person felt a study was overpowered in either framework.