I just want to buffer a little bit what you said about "the p-value is just checking a box more than being used as a tool for discovery.", but we are likely on

I just want to buffer a little bit what you said about "the p-value is just checking a box more than being used as a tool for discovery.", but we are likely on the same page. Sometimes, it is necessary to use a significance test in order to convince the reader that your conclusion based on your interpretation of your observations is sensical, which would be "checking a box". Other times, and as a frequent reviewer I do see that, infrequently, but still, where the authors see something where the only thing to see seems completely anecdotal (small N, large or fake error bars, no significance or ranking test), or to support their innacurate interpretation of a phenomenon (I see that with young and inexperienced graduate student in a young and inexperienced PI's lab). In these cases, having a proper experimental setup, sample size, and analysis, is part of the discovery. At the end of the day, it is important to make sure that science stays evidence-based and statistic is one of the tools that can assist, but only assist, in doing that.