4 ms·

Frequentists stats aren't wrong. It's just a special case that has been elevated to unreasonable standards. When the physical phenomenon in question is truly ra

Frequentists stats aren't wrong. It's just a special case that has been elevated to unreasonable standards. When the physical phenomenon in question is truly random, frequentist methods can be a convenient mathematical shortcut. But should we be teaching scientists the "shortcut"? Should we be forcing every publication to use these shortcuts? Statistic's role in the scientific reproducibility crisis says no.

kccqzy 1y ago
Frequentism methods are strictly less general. For example Laplace used probability theory to estimate the mass of Saturn. But with a frequentist interpretation we have to imagine a large number of parallel universes where everything remains the same except for the mass of Saturn. That's overly prescriptive of what probability means. Whereas in Bayesian statistics what probability means is strictly more general. You can manipulate probabilities even without fully defining them (maximum entropy) subject to intuitive rules (sum rule, product rule, Bayes' theorem), and the results of such manipulation are still correct and useful.
- deleted 1y ago
  [deleted]
- perrygeo 1y ago
  Drawing a sample of Saturns from an infinite set of Saturns! It's completely absurd, but that's what you get when you take a mathematical tool for coin flips and apply it to larger scientific questions. I wonder if the generality of the Bayesian approach is what's prevented its wide adoption? Having a prescribed algorithm ready to plug in data is mighty convenient! Frequentism lowered the barrier and let anyone run stats, but more isn't necessarily a good thing.
  - IshKebab 1y ago
    I dunno about you guys but I have no problems imagining randomly sampling Saturn.
    - mitthrowaway2 1y ago
      What do you mean by "randomly sampling" here?
      - IshKebab 1y ago
        I mean, Saturn was formed by some process right? And it must be sensitive to some initial conditions that - although maybe not really random, we can treat as random. Now imagine going back in time and changing those conditions a bit so that Saturn ended up differently. Do that 1000 times, giving you 1000 different Saturns. Now pick one randomly.
        chuckadams 1y ago
        We do that right now with computer simulations. Not exactly the hardest of evidence, but if the time machine were possible, someone in the future would have done it by now.
        NeutralCrane 1y ago
        The point is that you can’t do that. That’s the entire conundrum with Frequentism. They object to stating anything about the probability of Saturn, because from an objectivist point of view, any statement is either true or it isn’t, and therefore all probabilistic statements about it must be 0% or 100%. Instead they resort to statements about the frequencies over the long term from repeated processes, like the one you have. There are two problems with this: 1. They aren’t answering the original question. The question is about the probability of a property of Saturn. Not about the process of repeatedly forming thousands of alternative Saturns. This seems like a subtle difference but that’s only because Frequentism has been the default for so long. It doesn’t attempt to answer the questions people are actually asking. 2. The assumptions it makes to answer that alternative question are just as flawed. We can’t go back in time and change the conditions surrounding Saturn’s creation. We can’t run 1000s of repeated trials of the creation of Saturn. For a group of people so ideologically opposed to a statement as simple as “the probability of this flipped coin being heads is 50%”, it seems absurd that they are fine with their entire framework being built around a premise that doesn’t exist and cannot exist.
- StopDisinfo910 1y ago
  Laplace is typical use of inference statistics to built an estimator. I don’t really understand your point about parallel universe here. It’s absolutely not necessary for any of the sampling to make sense. Every time you try to measure anything, you are indeed taking a sample of the set of measures you could have gotten given the tools you are using. I fear you operate under the illusion that frequentist statistics are somehow limited to hypothesis testing. It is absolutely not the case.
- roenxi 1y ago
  > But with a frequentist interpretation we have to imagine a large number of parallel universes where everything remains the same except for the mass of Saturn. That's overly prescriptive of what probability means. That isn't much of an argument to the mathematicians. Nobody ever came up with a compelling explanation for what -1 sheeps look like and yet negative numbers turned out to be extremely practical. If it is absurd and provably works then the math community can roll with that.
  - kccqzy 1y ago
    It turns out -1 sheep is well defined: we use numbers to represent both absolute quantity and relative difference. A sheep died today, therefore the relative difference between today and yesterday is -1 sheep. Math people prefer to generalize. But with frequentism, it is not possible to generalize because "frequency" is baked into its very name. Indeed you can imagine Bayesian statistics as the generalization of frequentism.
wenc 1y ago
Frequentist methods are unintuitive and seemingly arbitrary to a beginner (hypothesis testing, 95% confidence, p=0.05). Bayesian methods are more intuitive, and fit how most be reason when they reason probabilistically. Unfortunately Bayesian computational methods are often less practical to use in non-trivial settings (usually involves some MCMC). I'm a Bayesian reasoner, but happily use frequentist computation methods (max likelihood estimation) because they're just more tractable.
- jampekka 1y ago
  p-value testing is problematic, but frequentist CIs typically map to credible intervals with uninformative priors. In practice in Bayesian analyses tend to use so weak priors that they are essentially uninformative. Maximum likelihood also tends to be equivalent to MAP with uninformative priors. I find a lot of Bayesian analysis is a bit of cargo culting and frequentist/ML formulations are dismissed with tribalism.
- porridgeraisin 1y ago
  I'm familiar with hypothesis testing, MCMC, and MLE. Can you explain how they are bayesian or frequentist?
tgv 1y ago
NHST, which is part of frequentist statistics, is wrong, plain and simple. It answers the wrong question (what's the probability of the data given the hypothesis vs. what's the probability of the hypothesis given the data), and will favor H1 under conditions that can be manipulated in advance. There is a total lack of understanding of how it works, but people think they know how to use it. There are numerous articles out there containing statements like "there were no differences in age between the groups (p > 0.05)". Consequently, it is the wrong thing to teach. That's apart from the more philosophical question: what does it mean when I say that there's a 40% chance that it team A will beat team B in the match tomorrow?
- StopDisinfo910 1y ago
  NHST is not wrong. It’s widely misused by people who barely understand any statistics. Reducing frequentist statistics to testing and p-value is a huge mistake. I have always wondered if that’s how it is introduced to some and that’s why they don’t get the point of the frequentist approach. Estimation theory makes a lot of sense - to me a lot more than pulling priors out of thin air. It’s also a lot of relatively advanced mathematics if you want to teach it well as defining random variables properly requires a fair bit of measure theory. I think the perceived gap comes from there. People have a somewhat hand wavy understanding of sampling and an overall poor grounding in theory and then think Bayes is better because it looks simpler at first.
  - zozbot234 1y ago
    > Estimation theory makes a lot of sense - to me a lot more than pulling priors out of thin air. You're "pulling priors out of thin air" whether you realize it or not; it's the only way that estimation makes sense mathematically. Frequentist statistics is broadly equivalent to Bayesian statistics with a flat prior distribution over the parameters, and what expectations correspond to a "flat" distribution ultimately depends on how the model is parameterized, which is in principle an arbitrary choice - something that's being "pulled out of thin air". Of course, Bayesian statistics also often involves assigning "uninformative" priors out of pure convenience, and frequentists can use "robust" statistical methods to exceptionally take prior information into account; so the difference is even lower than you might expect. There's also a strong argument against NHST specifically that works from both a frequentist and a Bayesian perspective: NHST rejects the Likelihood principle https://en.wikipedia.org/wiki/Likelihood_principle https://en.wikipedia.org/wiki/Likelihood_principle hence one could even ask whether NHST is even "properly" frequentist.
NeutralCrane 1y ago
Frequentist stats are wrong. They are built entirely on a flawed, non-consistent premise. The only reason for their use is because Bayesian approaches were computationally unfeasible for a long time, but that is no longer the case.

kccqzy 1y ago

Frequentism methods are strictly less general. For example Laplace used probability theory to estimate the mass of Saturn. But with a frequentist interpretation we have to imagine a large number of parallel universes where everything remains the same except for the mass of Saturn. That's overly prescriptive of what probability means. Whereas in Bayesian statistics what probability means is strictly more general. You can manipulate probabilities even without fully defining them (maximum entropy) subject to intuitive rules (sum rule, product rule, Bayes' theorem), and the results of such manipulation are still correct and useful.

deleted 1y ago
[deleted]
perrygeo 1y ago
Drawing a sample of Saturns from an infinite set of Saturns! It's completely absurd, but that's what you get when you take a mathematical tool for coin flips and apply it to larger scientific questions. I wonder if the generality of the Bayesian approach is what's prevented its wide adoption? Having a prescribed algorithm ready to plug in data is mighty convenient! Frequentism lowered the barrier and let anyone run stats, but more isn't necessarily a good thing.
- IshKebab 1y ago
  I dunno about you guys but I have no problems imagining randomly sampling Saturn.
  - mitthrowaway2 1y ago
    What do you mean by "randomly sampling" here?
    - IshKebab 1y ago
      I mean, Saturn was formed by some process right? And it must be sensitive to some initial conditions that - although maybe not really random, we can treat as random. Now imagine going back in time and changing those conditions a bit so that Saturn ended up differently. Do that 1000 times, giving you 1000 different Saturns. Now pick one randomly.
      - chuckadams 1y ago
        We do that right now with computer simulations. Not exactly the hardest of evidence, but if the time machine were possible, someone in the future would have done it by now.
      - NeutralCrane 1y ago
        The point is that you can’t do that. That’s the entire conundrum with Frequentism. They object to stating anything about the probability of Saturn, because from an objectivist point of view, any statement is either true or it isn’t, and therefore all probabilistic statements about it must be 0% or 100%. Instead they resort to statements about the frequencies over the long term from repeated processes, like the one you have. There are two problems with this: 1. They aren’t answering the original question. The question is about the probability of a property of Saturn. Not about the process of repeatedly forming thousands of alternative Saturns. This seems like a subtle difference but that’s only because Frequentism has been the default for so long. It doesn’t attempt to answer the questions people are actually asking. 2. The assumptions it makes to answer that alternative question are just as flawed. We can’t go back in time and change the conditions surrounding Saturn’s creation. We can’t run 1000s of repeated trials of the creation of Saturn. For a group of people so ideologically opposed to a statement as simple as “the probability of this flipped coin being heads is 50%”, it seems absurd that they are fine with their entire framework being built around a premise that doesn’t exist and cannot exist.
StopDisinfo910 1y ago
Laplace is typical use of inference statistics to built an estimator. I don’t really understand your point about parallel universe here. It’s absolutely not necessary for any of the sampling to make sense. Every time you try to measure anything, you are indeed taking a sample of the set of measures you could have gotten given the tools you are using. I fear you operate under the illusion that frequentist statistics are somehow limited to hypothesis testing. It is absolutely not the case.
roenxi 1y ago
> But with a frequentist interpretation we have to imagine a large number of parallel universes where everything remains the same except for the mass of Saturn. That's overly prescriptive of what probability means. That isn't much of an argument to the mathematicians. Nobody ever came up with a compelling explanation for what -1 sheeps look like and yet negative numbers turned out to be extremely practical. If it is absurd and provably works then the math community can roll with that.
- kccqzy 1y ago
  It turns out -1 sheep is well defined: we use numbers to represent both absolute quantity and relative difference. A sheep died today, therefore the relative difference between today and yesterday is -1 sheep. Math people prefer to generalize. But with frequentism, it is not possible to generalize because "frequency" is baked into its very name. Indeed you can imagine Bayesian statistics as the generalization of frequentism.

wenc 1y ago

Frequentist methods are unintuitive and seemingly arbitrary to a beginner (hypothesis testing, 95% confidence, p=0.05). Bayesian methods are more intuitive, and fit how most be reason when they reason probabilistically. Unfortunately Bayesian computational methods are often less practical to use in non-trivial settings (usually involves some MCMC). I'm a Bayesian reasoner, but happily use frequentist computation methods (max likelihood estimation) because they're just more tractable.

jampekka 1y ago
p-value testing is problematic, but frequentist CIs typically map to credible intervals with uninformative priors. In practice in Bayesian analyses tend to use so weak priors that they are essentially uninformative. Maximum likelihood also tends to be equivalent to MAP with uninformative priors. I find a lot of Bayesian analysis is a bit of cargo culting and frequentist/ML formulations are dismissed with tribalism.
porridgeraisin 1y ago
I'm familiar with hypothesis testing, MCMC, and MLE. Can you explain how they are bayesian or frequentist?

tgv 1y ago

NHST, which is part of frequentist statistics, is wrong, plain and simple. It answers the wrong question (what's the probability of the data given the hypothesis vs. what's the probability of the hypothesis given the data), and will favor H1 under conditions that can be manipulated in advance. There is a total lack of understanding of how it works, but people think they know how to use it. There are numerous articles out there containing statements like "there were no differences in age between the groups (p > 0.05)". Consequently, it is the wrong thing to teach. That's apart from the more philosophical question: what does it mean when I say that there's a 40% chance that it team A will beat team B in the match tomorrow?

StopDisinfo910 1y ago
NHST is not wrong. It’s widely misused by people who barely understand any statistics. Reducing frequentist statistics to testing and p-value is a huge mistake. I have always wondered if that’s how it is introduced to some and that’s why they don’t get the point of the frequentist approach. Estimation theory makes a lot of sense - to me a lot more than pulling priors out of thin air. It’s also a lot of relatively advanced mathematics if you want to teach it well as defining random variables properly requires a fair bit of measure theory. I think the perceived gap comes from there. People have a somewhat hand wavy understanding of sampling and an overall poor grounding in theory and then think Bayes is better because it looks simpler at first.
- zozbot234 1y ago
  > Estimation theory makes a lot of sense - to me a lot more than pulling priors out of thin air. You're "pulling priors out of thin air" whether you realize it or not; it's the only way that estimation makes sense mathematically. Frequentist statistics is broadly equivalent to Bayesian statistics with a flat prior distribution over the parameters, and what expectations correspond to a "flat" distribution ultimately depends on how the model is parameterized, which is in principle an arbitrary choice - something that's being "pulled out of thin air". Of course, Bayesian statistics also often involves assigning "uninformative" priors out of pure convenience, and frequentists can use "robust" statistical methods to exceptionally take prior information into account; so the difference is even lower than you might expect. There's also a strong argument against NHST specifically that works from both a frequentist and a Bayesian perspective: NHST rejects the Likelihood principle https://en.wikipedia.org/wiki/Likelihood_principle https://en.wikipedia.org/wiki/Likelihood_principle hence one could even ask whether NHST is even "properly" frequentist.

NeutralCrane 1y ago

Frequentist stats are wrong. They are built entirely on a flawed, non-consistent premise. The only reason for their use is because Bayesian approaches were computationally unfeasible for a long time, but that is no longer the case.