In practice, people, even scientists, treat smaller p-values as better "evidence". That's not what p-vals mean, and even if it were, there are biases in p-vals

In practice, people, even scientists, treat smaller p-values as better "evidence". That's not what p-vals mean, and even if it were, there are biases in p-vals that make them problematic as the number of data points increase. Bayesian stats aren't related to training and test sets, like in machine learning; scientific experiments rarely partition data like that. A Bayesian analysis answers the question, "Given the evidence I just collected, how should I adjust my estimate of how likely something is?" But Bayesian analyses have issues too: the biggest is not knowing what your initial, "prior" probability should be. (After all, that's usually what you're trying to find out!) With something simple like a coin flip, you have a strong prior assumption of 50% probability of heads. And neither Bayesian nor frequentist methods address effect size. If you collect enough data, you can detect extremely small (real) effect sizes that pass statistical tests, but are still meaningless outside the context of publishing a paper. Rather than analysis type, we should incorporate more discussion of effect size, especially as the number of data points increase.