5 ms·

Frequentist is the default. People don’t promote it. They just teach it as statistics, use p-values, and look funny at anyone who does otherwise. It’s like how

Frequentist is the default. People don’t promote it. They just teach it as statistics, use p-values, and look funny at anyone who does otherwise. It’s like how you wouldn’t hear people in the US promote imperial measurements for home baking. When you dominate, you don’t have to advocate.

analog31 3y ago
I'd identify a larger problem, which is teaching anything as a bunch of recipes. My college offered stats in two tracks: There was a one-semester course for science majors, which was mostly plugging numbers into formulas. The course was utterly baffling for most of the students who took it. There was also a two-semester course for math majors. It was mainly about proofs, but also had time to go into more depth. You have to know the assumptions underlying a formula, if you're expected to prove it. ;-) But it was only taken by the math majors.
- deleted 3y ago
  [deleted]
- dan-robertson 3y ago
  I’m not sure that teaching mathstats more broadly is the answer to frequentist approaches being used by default, or to the recipe problem. I did the whole mathstats thing as a student and spent a bunch of time doing calculus around things like maximum likelihood estimators or unbiased estimators. I don’t think it’s very useful for the application of statistics in practice. I would recommend Statistical Rethinking by Richard McElreath as a course in Bayesian statistics targeted at scientists who want to extract correct, useful conclusions from their experiments, which doesn’t follow the “recipe book” pattern.
  - analog31 3y ago
    I'm not sure of that either. For one thing, the students would be blown away. One thing we didn't have when I took stats was computers. I graduated from high school in 1982, and the colleges in my state were just beginning to get computers. I wonder if stats could be approached differently if it could start with nothing but data -- lots of it -- and graphing tools. This could even happen pre-college.
  - lmpdev 3y ago
    I don’t know if it’s the “answer”, but it followed a pretty simple logistic curve for our curriculum at uni First course was almost purely frequentist Second was half and half with several weeks approaching the frequentist bayesian approach from both sides Our third stats course was mostly bayesian though self directed as to which point of view was “most appropriate” for the formulated problem sets. I remember roughly half the cohort used a frequentist perspective on a fairly obvious (but not explicit) bayesian assignment and were severely punished by the course coordinator (fairly for a late year subject tbh)
mjfl 3y ago
But I don't see p-values as connected to frequentist statistics. Rather, p-values are a concept that applies to the problem of hypothesis testing, which both frequentist and bayesian statistics should apply to, right? And in both cases you would use a p-value. The unique thing that Bayesian statistics can do is fit parameters can calculate uncertainty in those parameters, which does not contradict any of the concepts of hypothesis testing.
- travisjungroth 3y ago
  A p-value is the frequency you’d expect to see a sample with an effect this size or greater, given the null hypothesis. It’s inherently a frequentist approach to inference. You can squish it into Bayes by considering it a uniform prior on the real number line that you never update, but you’re not really doing things “in the spirit” of Bayes, then.
  - mjfl 3y ago
    What is a “prior” in the context of hypothesis testing? That seems to me to be a concept used in parameter estimation, with Bayes rule. But hypothesis testing is not parameter estimation.
    - travisjungroth 3y ago
      As the linked post points out, you can regularize a p-value based on prior experiments. IMO, null hypothesis testing is way overused. We should be quantifying the comparison of the null versus alternative hypotheses. People already compare them. Might as well bring some math into it.
    - mitthrowaway2 3y ago
      How is hypothesis testing different from parameter estimation? They seem conceptually the same to me, with one a special case of the other.
    - PheonixPharts 3y ago
      In Bayesian terms there is no distinction between hypothesis testing and parameter estimation. If your hypothesis is that A is greater than B, then you're test boils down to "The probability that A is greater than B", which is arrived at through parameter estimation via Bayes' Theorem. You can consider the distribution of estimates for a parameter to represent a space of possible hypothesis for the true value of that parameter and their absolutely or relative likelihood based on the information available. The also has the pleasant consequence that hypothesis testing using Bayesian methods nearly always is closer to the actual question someone wants to answer rather than replying with confusing statements such as "the probability of rejecting the null hypothesis", which contains an implicit double negative.

analog31 3y ago

I'd identify a larger problem, which is teaching anything as a bunch of recipes. My college offered stats in two tracks: There was a one-semester course for science majors, which was mostly plugging numbers into formulas. The course was utterly baffling for most of the students who took it. There was also a two-semester course for math majors. It was mainly about proofs, but also had time to go into more depth. You have to know the assumptions underlying a formula, if you're expected to prove it. ;-) But it was only taken by the math majors.

deleted 3y ago
[deleted]
dan-robertson 3y ago
I’m not sure that teaching mathstats more broadly is the answer to frequentist approaches being used by default, or to the recipe problem. I did the whole mathstats thing as a student and spent a bunch of time doing calculus around things like maximum likelihood estimators or unbiased estimators. I don’t think it’s very useful for the application of statistics in practice. I would recommend Statistical Rethinking by Richard McElreath as a course in Bayesian statistics targeted at scientists who want to extract correct, useful conclusions from their experiments, which doesn’t follow the “recipe book” pattern.
- analog31 3y ago
  I'm not sure of that either. For one thing, the students would be blown away. One thing we didn't have when I took stats was computers. I graduated from high school in 1982, and the colleges in my state were just beginning to get computers. I wonder if stats could be approached differently if it could start with nothing but data -- lots of it -- and graphing tools. This could even happen pre-college.
- lmpdev 3y ago
  I don’t know if it’s the “answer”, but it followed a pretty simple logistic curve for our curriculum at uni First course was almost purely frequentist Second was half and half with several weeks approaching the frequentist bayesian approach from both sides Our third stats course was mostly bayesian though self directed as to which point of view was “most appropriate” for the formulated problem sets. I remember roughly half the cohort used a frequentist perspective on a fairly obvious (but not explicit) bayesian assignment and were severely punished by the course coordinator (fairly for a late year subject tbh)

mjfl 3y ago

But I don't see p-values as connected to frequentist statistics. Rather, p-values are a concept that applies to the problem of hypothesis testing, which both frequentist and bayesian statistics should apply to, right? And in both cases you would use a p-value. The unique thing that Bayesian statistics can do is fit parameters can calculate uncertainty in those parameters, which does not contradict any of the concepts of hypothesis testing.

travisjungroth 3y ago
A p-value is the frequency you’d expect to see a sample with an effect this size or greater, given the null hypothesis. It’s inherently a frequentist approach to inference. You can squish it into Bayes by considering it a uniform prior on the real number line that you never update, but you’re not really doing things “in the spirit” of Bayes, then.
- mjfl 3y ago
  What is a “prior” in the context of hypothesis testing? That seems to me to be a concept used in parameter estimation, with Bayes rule. But hypothesis testing is not parameter estimation.
  - travisjungroth 3y ago
    As the linked post points out, you can regularize a p-value based on prior experiments. IMO, null hypothesis testing is way overused. We should be quantifying the comparison of the null versus alternative hypotheses. People already compare them. Might as well bring some math into it.
  - mitthrowaway2 3y ago
    How is hypothesis testing different from parameter estimation? They seem conceptually the same to me, with one a special case of the other.
  - PheonixPharts 3y ago
    In Bayesian terms there is no distinction between hypothesis testing and parameter estimation. If your hypothesis is that A is greater than B, then you're test boils down to "The probability that A is greater than B", which is arrived at through parameter estimation via Bayes' Theorem. You can consider the distribution of estimates for a parameter to represent a space of possible hypothesis for the true value of that parameter and their absolutely or relative likelihood based on the information available. The also has the pleasant consequence that hypothesis testing using Bayesian methods nearly always is closer to the actual question someone wants to answer rather than replying with confusing statements such as "the probability of rejecting the null hypothesis", which contains an implicit double negative.