4 ms·

> I've never heard anyone promote frequentist statistics I will do it: 1. Computationally easier 2. often analytical theory available for most use cases so i

> I've never heard anyone promote frequentist statistics I will do it: 1. Computationally easier 2. often analytical theory available for most use cases so interpretability is high 3. more literature available so you can get unstuck faster if you mess up 4. no accusations of subjective bias in your prior (the con is clear, no ability to leverage subjective expertise) 5. In the asymptotic regime, MLE and bayesian MAP often converge anyways 6. king of hypothesis testing For most people, it doesn't matter. It matters when you are doing treatment for small sample sizes or other situations that would cause low power.

bogtog 3y ago
> king of hypothesis testing This advantage can't be understated. Researchers, at least in psychology, almost never seem to care about the magnitude of an effect. It's simply enough to show that some effect happens in some direction. For this (generally acceptable) purpose, frequentist stats are great.
- urschrei 3y ago
  A statistically significant result without something like a Hedges G (or Cohens D if comparing means) measure would be called out by even a moderately attentive reviewer as a matter of course (especially if small sample sizes are involved), and many reputable journals actually require effect sizes to be stated, so you run the risk of being desk rejected for leaving it out.
  - nullc 3y ago
    For leaving it out perhaps, but the result is publishable even if the effect is small... where for significant you usually must p-hack your way to a standard threshold. It's not entirely unreasonable, as often extra variables suppress the effect size. E.g. the effect is substantial in people with some genetics, insignificant otherwise. The fact that there is an effect at all makes it interesting as a starting point for further exploration e.g. to determine the mechanism or find the extra factors needed to make the effect significant.
- travisjungroth 3y ago
  Add some sampling bias, a significance threshold of 0.05. Baby, you’ve got a replication crisis going!
- naasking 3y ago
  > This advantage can't be understated. Researchers, at least in psychology, almost never seem to care about the magnitude of an effect. It's simply enough to show that some effect happens in some direction That's not an advantage, that's one of the primary reasons psychology is among the worst subjects hit by the replication crisis.
  - SubiculumCode 3y ago
    A cheap shot that misunderstands the so called "crisis" in psychology, which almost comes entirely from the fact that brains are non randomly exchangeable, unique, samples differ, the brain is complex, multiple ways to solve problems, and sussing it all out is complex as hell.
    - naasking 3y ago
      If they are at least as knowledgeable as you, who correctly describes how messy you can expect the data to be, then you should be more concerned with effect size, because small effects are almost certainly just "noise". Instead, sensationalism about miniscule effects that vanish on replication attempts abound. So not a cheap shot at all IMO.
  - bogtog 3y ago
    Sorry for this very slow response, but I actually disagree. I don’t think effect sizes are usually meaningful beyond the context of a specific experiment. For example, suppose you are researching the effect of an emotional/negative picture on how a participant makes decision in an economic game. Here, you may think “a big effect size implies this study will have a large practical relevance.” However, following typical statistical designs, an effect size may be largely determined by just how many trials the experiment used for each participant. This is also not the only factor that’s relevant. For instance, it’s debatable how the emotion induced in the task compares to the intensity of real life emotional situations. With these factors in mind, the effect size really doesn’t tell you much that can be applied outside the laboratory. Maybe in some non-experimental sub-fields (e.g. personality psychology) effect sizes are more meaningful…

bogtog 3y ago

> king of hypothesis testing This advantage can't be understated. Researchers, at least in psychology, almost never seem to care about the magnitude of an effect. It's simply enough to show that some effect happens in some direction. For this (generally acceptable) purpose, frequentist stats are great.

urschrei 3y ago
A statistically significant result without something like a Hedges G (or Cohens D if comparing means) measure would be called out by even a moderately attentive reviewer as a matter of course (especially if small sample sizes are involved), and many reputable journals actually require effect sizes to be stated, so you run the risk of being desk rejected for leaving it out.
- nullc 3y ago
  For leaving it out perhaps, but the result is publishable even if the effect is small... where for significant you usually must p-hack your way to a standard threshold. It's not entirely unreasonable, as often extra variables suppress the effect size. E.g. the effect is substantial in people with some genetics, insignificant otherwise. The fact that there is an effect at all makes it interesting as a starting point for further exploration e.g. to determine the mechanism or find the extra factors needed to make the effect significant.
travisjungroth 3y ago
Add some sampling bias, a significance threshold of 0.05. Baby, you’ve got a replication crisis going!
naasking 3y ago
> This advantage can't be understated. Researchers, at least in psychology, almost never seem to care about the magnitude of an effect. It's simply enough to show that some effect happens in some direction That's not an advantage, that's one of the primary reasons psychology is among the worst subjects hit by the replication crisis.
- SubiculumCode 3y ago
  A cheap shot that misunderstands the so called "crisis" in psychology, which almost comes entirely from the fact that brains are non randomly exchangeable, unique, samples differ, the brain is complex, multiple ways to solve problems, and sussing it all out is complex as hell.
  - naasking 3y ago
    If they are at least as knowledgeable as you, who correctly describes how messy you can expect the data to be, then you should be more concerned with effect size, because small effects are almost certainly just "noise". Instead, sensationalism about miniscule effects that vanish on replication attempts abound. So not a cheap shot at all IMO.
- bogtog 3y ago
  Sorry for this very slow response, but I actually disagree. I don’t think effect sizes are usually meaningful beyond the context of a specific experiment. For example, suppose you are researching the effect of an emotional/negative picture on how a participant makes decision in an economic game. Here, you may think “a big effect size implies this study will have a large practical relevance.” However, following typical statistical designs, an effect size may be largely determined by just how many trials the experiment used for each participant. This is also not the only factor that’s relevant. For instance, it’s debatable how the emotion induced in the task compares to the intensity of real life emotional situations. With these factors in mind, the effect size really doesn’t tell you much that can be applied outside the laboratory. Maybe in some non-experimental sub-fields (e.g. personality psychology) effect sizes are more meaningful…