Great catch. This is a super important factor. The real concern in science is how easy it can be to hide the fact that so many possibilities were tested, preclu

Great catch. This is a super important factor. The real concern in science is how easy it can be to hide the fact that so many possibilities were tested, precluding this observation. As another example, there are GWAS studies where people take surveys and medical histories of large groups of people and then sequence their DNA to look for correlations across all possibilities. Those studies only report results with p values less than 1e-4 or 1e-5.