These are reasonable criteria. I also tend to be very sensitive to failure to correct for multiple hypotheses, as this is something I see all the time, particu

These are reasonable criteria. I also tend to be very sensitive to failure to correct for multiple hypotheses, as this is something I see all the time, particularly when people start sub-setting data: "We looked for an association between vegetables in the diet and cancer incidence, but only found it between kale and lung cancer." This happens all the time, and people report such associations as if they were the only experiment being run, whereas in fact they have run some combinatorically huge list of alternative hypotheses, and unsurprisingly have found one that looks significant at p=0.05 (which is a ridiculously lax acceptance criterion.) I also pretty much categorically reject case control studies: http://www.tjradcliffe.com/?p=1745 http://www.tjradcliffe.com/?p=1745 They are insanely over-sensitive to confounding factors. They can and do have legitimate uses to guide further research, but should never be used as the basis of policy or action beyond that. There's also a sense one gets from many papers that the researchers are black-boxing their statistical analysis: that they have plugged the numbers into some standard package and take the results at face value. While I appreciate that maybe not everyone can have a solid technical grasp of this stuff, it always bothers me when I see that because it is far too easy to generate garbage if you don't understand precisely what you're doing. [Disclaimer: I am an experimental and computational physicist who has never taken a stats course, but believe myself to be competently self-educated in the subject and have spent part of my career doing data analysis professionally using primarily Bayesian methods.]