>"It's crazy that by using a p-value of 0.05, it means that 5% of all scientific results might be false." How so? I don't see any connection between significan

>"It's crazy that by using a p-value of 0.05, it means that 5% of all scientific results might be false." How so? I don't see any connection between significance level and % of false scientific results at all. If you assume the "null hypothesis" is always true, then 5% of the results should falsely say otherwise. Of course, this is if all the assumptions behind the math hold, no p-hacking, etc. However, that is like saying it is extremely rare for there to be a correlation between any two phenomena. We don't live in that universe. In our universe, correlations are extremely common: >"These armchair considerations are borne out by the finding that in psychological and sociological investigations involving very large numbers of subjects, it is regularly found that almost all correlations or differences between means are statistically significant. See, for example, the papers by Bakan [1] and Nunnally [8]. Data currently being analyzed by Dr. David Lykken and myself, derived from a huge sample of over 55,000 Minnesota high school seniors, reveal statistically significant relationships in 91% of pairwise associations among a congeries of 45 miscellaneous variables such as sex, birth order, religious preference, number of siblings, vocational choice, club membership, college choice, mother's education, dancing, interest in woodworking, liking for school, and the like. The 9% of non-significant associations are heavily concentrated among a small minority of variables having dubious reliability, or involving arbitrary groupings of non-homogeneous or nonmonotonic sub-categories. The majority of variables exhibited significant relationships with all but three of the others, often at a very high confidence level" -Theory testing in psychology and physics: A methodological paradox. http://www.fisme.science.uu.nl/staff/christianb/downloads/meehl1967.pdf http://www.fisme.science.uu.nl/staff/christianb/downloads/me...