3 ms·

> This advantage can't be understated. Researchers, at least in psychology, almost never seem to care about the magnitude of an effect. It's simply enough to sh

> This advantage can't be understated. Researchers, at least in psychology, almost never seem to care about the magnitude of an effect. It's simply enough to show that some effect happens in some direction That's not an advantage, that's one of the primary reasons psychology is among the worst subjects hit by the replication crisis.

SubiculumCode 3y ago
A cheap shot that misunderstands the so called "crisis" in psychology, which almost comes entirely from the fact that brains are non randomly exchangeable, unique, samples differ, the brain is complex, multiple ways to solve problems, and sussing it all out is complex as hell.
- naasking 3y ago
  If they are at least as knowledgeable as you, who correctly describes how messy you can expect the data to be, then you should be more concerned with effect size, because small effects are almost certainly just "noise". Instead, sensationalism about miniscule effects that vanish on replication attempts abound. So not a cheap shot at all IMO.
bogtog 3y ago
Sorry for this very slow response, but I actually disagree. I don’t think effect sizes are usually meaningful beyond the context of a specific experiment. For example, suppose you are researching the effect of an emotional/negative picture on how a participant makes decision in an economic game. Here, you may think “a big effect size implies this study will have a large practical relevance.” However, following typical statistical designs, an effect size may be largely determined by just how many trials the experiment used for each participant. This is also not the only factor that’s relevant. For instance, it’s debatable how the emotion induced in the task compares to the intensity of real life emotional situations. With these factors in mind, the effect size really doesn’t tell you much that can be applied outside the laboratory. Maybe in some non-experimental sub-fields (e.g. personality psychology) effect sizes are more meaningful…

SubiculumCode 3y ago

A cheap shot that misunderstands the so called "crisis" in psychology, which almost comes entirely from the fact that brains are non randomly exchangeable, unique, samples differ, the brain is complex, multiple ways to solve problems, and sussing it all out is complex as hell.

naasking 3y ago
If they are at least as knowledgeable as you, who correctly describes how messy you can expect the data to be, then you should be more concerned with effect size, because small effects are almost certainly just "noise". Instead, sensationalism about miniscule effects that vanish on replication attempts abound. So not a cheap shot at all IMO.

bogtog 3y ago

Sorry for this very slow response, but I actually disagree. I don’t think effect sizes are usually meaningful beyond the context of a specific experiment. For example, suppose you are researching the effect of an emotional/negative picture on how a participant makes decision in an economic game. Here, you may think “a big effect size implies this study will have a large practical relevance.” However, following typical statistical designs, an effect size may be largely determined by just how many trials the experiment used for each participant. This is also not the only factor that’s relevant. For instance, it’s debatable how the emotion induced in the task compares to the intensity of real life emotional situations. With these factors in mind, the effect size really doesn’t tell you much that can be applied outside the laboratory. Maybe in some non-experimental sub-fields (e.g. personality psychology) effect sizes are more meaningful…