A science writer starts off his article about science writing. He describes them being "surprised" about a meta-analysis attempting to reproduce scientific res

A science writer starts off his article about science writing. He describes them being "surprised" about a meta-analysis attempting to reproduce scientific results in a "fuzzy" scientific field (my definition of fuzzy science is any that studies humans). They can only reproduce 33% of the experiments. What a shocker (not really). It is common in the hard sciences to make fun of the "findings" in the fuzzy sciences. Humans are fickle creatures with a large variance in almost everything we do. With large variances, you have to collect large samples to be confident of your results. But as previously mentioned, humans are fickle, and it is difficult to get large statistically relevant samples of humans. Paying volunteers increases your sample size, but can bias your results towards a certain type of person. Not paying volunteers may cause them not to treat an experiment seriously. Gathering a large group of people together to make the study go faster may cause some word of the study to leak out and some group think to start. Some people just say what they think the researchers want to hear. Doing "science" with humans is incredibly difficult. So really, it's not surprising that one off experiments are hard to reproduce. Their sample sizes are small and can't adequately account for variance across a population of people. And results for region of the world doesn't often generalize to other cultures. My rule of thumb for trusting fuzzy data is only to start really believing it when there are several studies looking from several different angles and they all come to similar conclusions. If the studies contradict some previously held knowledge, it will take several more studies than forming a conclusion from scratch. But this is how science works, and isn't really surprising.