4 ms·
Every time this topic comes up I'm reminded of what Stefan Savage, a hero of mine, said about academic papers ("studies", in the sense we're discussing here): t
by tptacek 2y ago
Every time this topic comes up I'm reminded of what Stefan Savage, a hero of mine, said about academic papers ("studies", in the sense we're discussing here): they are the beginnings of conversations, not the end. It shouldn't shock people that results in papers might not replicate; papers aren't infallible, which makes sense when you consider the process that produces them.
- robwwilliams 2y agoThat is a generous interpretation. But in many cases we try our best to dress up studies and tell good stories—-preferably stories with compelling positive statistics and with slick figures. The story telling often obscures the key data.
- mrguyorama 2y agoNo, "Science" "Journalists" who have zero scientific training will take an extremely limited study and completely miss the point and make wild claims that the general population becomes convinced the "science" said. Scientists do not consider a study to be more than an observation. What matters to scientists is the totality of the evidence. There are several chemistry youtubers who have failed to "reproduce" papers they are working from. Does that mean chemistry is a farce and doesn't work? No, it means some chemist at some point in history failed to write something down, mostly because they didn't even know or realize it mattered. Science is incredibly difficult and we can only hope to be okay at it.
- tptacek 2y agoJust keep in mind while we're scare-quoting "journalists" that the scientists on HN also seem to think a failure of replication in a paper is a devastating indictment of whole scientific fields. It's not like we're that much better.
- robwwilliams 2y agoBoth ways. I have published several hundred peer-reviewed papers in biomedical research over 40 years. My experience is that almost all papers are written to maximize impact on the minds of readers—-ideally hewing to the data and limits of the design. But often the story telling aspects of a paper and the data snd design do not see eye-to-eye.
- oxym0ron 2y agoYes, papers start conversations, not end them. Replication issues are part of the academic process.