4 ms·
Useful background, if you aren't familiar: https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis Noteworthy is tha
by bell-cot 4y ago
Useful background, if you aren't familiar:
https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis
Noteworthy is that the crisis is a huge deal in Psychology - a field which "real" scientists were sneering at a century or more ago.
Personal anecdote:
I once knew a guy who majored in Psychology at a pretty prestigious U.S. research university, back in the mid 1980's. He said that the Psych Dept. there did a big survey of Psych undergrads, asking what they thought of the subject. The most significant finding? That the Psych majors thought the first 2 years of Psych classes were real facts about the real world. But after that - they thought that it was all bullsh*t, and learning how create and spew bullsh*t yourself. (The guy went on to law school, and was quite successful. Which could be interpreted in interesting ways.)
- mattkrause 4y agoThat sneering misses the mark, IMO. Other fields definitely have similar problems: Amgen found that they could only replicate about half of the cancer biology papers they tested. The "replication crisis" showed up in psychology first for a few reasons. First, psychology probably runs the most replications of any field. The experiments themselves are easy to reproduce: all you need to redo a survey project is a Qualtrics account—--or a photocopier. They can often be done fairly quickly, making replications a good "warm-up" project for a new grad student. The field also seems to value replications, at least a little. Second, psychology is actually quite hard! Some failed replications surely reflect sloppy or sketchy research practices, but many of them are probably due to reflect uncontrolled variables/variance. A physicist can produce an endless stream of photons, all of which are exact copies and totally independent of each other. Psychologists have nothing like that. People's reactions are shaped by their own idiosyncratic biases, their past history, and even what they think the experimenter wants to see. It's often not clear what variables actually matter, and in that sense, "failed" replications are sometimes more interesting than successes.
- 3pt14159 4y agoNot going to comment on psychology directly, because I don't know enough about it, but there are some fields where there really is no replication crisis. In structural engineering we literally tore metals apart to find their physical properties. Concrete was mixed. Giant crushing machines were used. Etc. Sometimes I think that half the reason why some fields don't have a replication crisis is because literal lives are on the line if you get it wrong, so people actually do the work to make the testing gear and align the incentives that need to be aligned.
- MichaelZuo 4y agoThere's definitely no replication crisis in jet turbine design research also.
- mattkrause 4y agoahem https://retractionwatch.com/?s=turbine https://retractionwatch.com/?s=turbine A lot of these appear to be cases of flat-out fraud or dubious research practices. These all get muddled together under the heading of "replication crisis" but I do think they're a bit of a different beast.
- MichaelZuo 4y agoYou linked to something about wind turbines. Did you link to the wrong article?
- thelamest 4y agoYeah, yet this bonduary traces a fuzzy and surprising route. Lives are on the line in microbiology, yet Bik uncovered thousands of suspect image errors there (https://en.wikipedia.org/wiki/Elisabeth_Bik https://en.wikipedia.org/wiki/Elisabeth_Bik). Semiconductor physics is relatively tangible and close to production, yet Hsu&Loo and others found that Schön faked his award and attention grabbing recipes (https://en.wikipedia.org/wiki/Sch%C3%B6n_scandal https://en.wikipedia.org/wiki/Sch%C3%B6n_scandal).
- mattkrause 4y ago