4 ms·

The problem is that replication studies aren't done hard enough. And by replication studies I mean "take this paper, try to replicate, write a paper on it". E

by p_l 3y ago

The problem is that replication studies aren't done hard enough. And by replication studies I mean "take this paper, try to replicate, write a paper on it". Every singular paper can be wrong, that's why you're supposed to replicate them - that's at least what was taught in psychology, which many consider "social science of less rigour" - well, it was probably the course most hardcore about experimental design in my education and ability to replicate a study...

maximus-decimus 3y ago
Are they not done or are they done and nobody wants to publish them?
- neuronerdgirl 3y ago
  Nobody CAN publish them. An actual criterion for most top journals is literally "novelty". Reviewers will immediately levee "this has been done before" as a damning critique in peer review. There is no replication because you can't currently share replications, let alone sustain a career on them.
zilti 3y ago
My experience of "representative" studies in psychology (university of Berne) was that they'd come up with something, then gather a group of "volunteer" psychology students - 80% women - who need to participate at x studies as requirement to pass, publish it as representative serious research, and call it a day.
the_snooze 3y ago
>The problem is that replication studies aren't done hard enough. >And by replication studies I mean "take this paper, try to replicate, write a paper on it". You don't necessarily need to perform a straight-up "replication study" in order to do some useful sanity checking on published results. It can be done as part of another novel study that builds upon prior work. That's basically what happened with the fraudulent UCLA political persuasion study several years ago [1]. Some Berkeley researchers wanted to extend the ideas in that earlier published paper, reused similar methods under the assumption that the UCLA folks were correct, and couldn't get anything reasonable within the ballpark of what was published. [1] https://fivethirtyeight.com/features/how-two-grad-students-uncovered-michael-lacour-fraud-and-a-way-to-change-opinions-on-transgender-rights/ https://fivethirtyeight.com/features/how-two-grad-students-u...

by p_l 3y ago

maximus-decimus 3y ago

Are they not done or are they done and nobody wants to publish them?

neuronerdgirl 3y ago
Nobody CAN publish them. An actual criterion for most top journals is literally "novelty". Reviewers will immediately levee "this has been done before" as a damning critique in peer review. There is no replication because you can't currently share replications, let alone sustain a career on them.

zilti 3y ago

My experience of "representative" studies in psychology (university of Berne) was that they'd come up with something, then gather a group of "volunteer" psychology students - 80% women - who need to participate at x studies as requirement to pass, publish it as representative serious research, and call it a day.

the_snooze 3y ago

>The problem is that replication studies aren't done hard enough. >And by replication studies I mean "take this paper, try to replicate, write a paper on it". You don't necessarily need to perform a straight-up "replication study" in order to do some useful sanity checking on published results. It can be done as part of another novel study that builds upon prior work. That's basically what happened with the fraudulent UCLA political persuasion study several years ago [1]. Some Berkeley researchers wanted to extend the ideas in that earlier published paper, reused similar methods under the assumption that the UCLA folks were correct, and couldn't get anything reasonable within the ballpark of what was published. [1] https://fivethirtyeight.com/features/how-two-grad-students-uncovered-michael-lacour-fraud-and-a-way-to-change-opinions-on-transgender-rights/ https://fivethirtyeight.com/features/how-two-grad-students-u...