3 ms·

hard to accidentally glue a frequentist model together with a prior ;)

kgwgk 2y ago
Also hard to interpret correctly frequentist results. -- Misinterpretations of P-values and statistical tests persists among researchers and professionals working with statistics and epidemiology "Correct inferences to both questions, which is that a statistically significant finding cannot be inferred as either proof or a measure of a hypothesis’ probability, were given by 10.7% of doctoral students and 12.5% of statisticians/epidemiologists." https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9383044/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9383044/ -- Robust misinterpretation of confidence intervals "Only 8 first-year students (2%), no master students, and 3 postmasters researchers (3%) correctly indicated that all statements were wrong." https://link.springer.com/article/10.3758/s13423-013-0572-3 https://link.springer.com/article/10.3758/s13423-013-0572-3 -- P-Value, Confidence Intervals, and Statistical Inference: A New Dataset of Misinterpretation "The data indicates that 99% subjects have at least 1 wrong answer of P-value understanding (Figure 1A) and 93% subjects have at least 1 wrong answer of CI understanding (Figure 1B)." https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2018.00868/full https://www.frontiersin.org/journals/psychology/articles/10....
ants_everywhere 2y ago
Oh it happens all the time. I've been in several lab meetings where the experiment was redesigned because the results came out "wrong." I.e. the (frequentist) statistics didn't match with the (implicit) prior.
- bayesian_trout 2y ago
  This is not a statistics problem, but instead an ethics problem, ha.
  - ants_everywhere 2y ago
    I agree totally. But it's also a statistics problem because ethically you should incorporate your assumptions into the model. If the assumptions are statistical, then you can incorporate them in a prior.
    - bayesian_trout 2y ago
      I mean, the biggest assumptions that most influence the inferences one makes are rarely "statistical" in the sense that they can actually be incorporated in a particular analysis via a prior. They tend to be structural assumptions that represent some fundamental limit to your current state of knowledge, no? Certainly this is domain-specific, though. I once read a Gelman blog post or paper that argued Frequentists should be more Frequentist (i.e., repeat experiments more often than they currently do) and Bayesians should be more Bayesian (i.e., be more willing to use informative priors and or make probability statements beyond 95% credible intervals). Or something like that, as I am paraphrasing. That always seemed reasonable. Either way, the dueling--and highly simplified--caricatures of Bayesians vs. Frequentists vs. likelihood folks is largely silly to me. Use the tool that works best for the job at hand, and if you can answer a problem effectively with a well designed experiment and a t-test so be it.

Also hard to interpret correctly frequentist results. -- Misinterpretations of P-values and statistical tests persists among researchers and professionals working with statistics and epidemiology "Correct inferences to both questions, which is that a statistically significant finding cannot be inferred as either proof or a measure of a hypothesis’ probability, were given by 10.7% of doctoral students and 12.5% of statisticians/epidemiologists." https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9383044/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9383044/ -- Robust misinterpretation of confidence intervals "Only 8 first-year students (2%), no master students, and 3 postmasters researchers (3%) correctly indicated that all statements were wrong." https://link.springer.com/article/10.3758/s13423-013-0572-3 https://link.springer.com/article/10.3758/s13423-013-0572-3 -- P-Value, Confidence Intervals, and Statistical Inference: A New Dataset of Misinterpretation "The data indicates that 99% subjects have at least 1 wrong answer of P-value understanding (Figure 1A) and 93% subjects have at least 1 wrong answer of CI understanding (Figure 1B)." https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2018.00868/full https://www.frontiersin.org/journals/psychology/articles/10....

ants_everywhere 2y ago

Oh it happens all the time. I've been in several lab meetings where the experiment was redesigned because the results came out "wrong." I.e. the (frequentist) statistics didn't match with the (implicit) prior.

bayesian_trout 2y ago
This is not a statistics problem, but instead an ethics problem, ha.
- ants_everywhere 2y ago
  I agree totally. But it's also a statistics problem because ethically you should incorporate your assumptions into the model. If the assumptions are statistical, then you can incorporate them in a prior.
  - bayesian_trout 2y ago
    I mean, the biggest assumptions that most influence the inferences one makes are rarely "statistical" in the sense that they can actually be incorporated in a particular analysis via a prior. They tend to be structural assumptions that represent some fundamental limit to your current state of knowledge, no? Certainly this is domain-specific, though. I once read a Gelman blog post or paper that argued Frequentists should be more Frequentist (i.e., repeat experiments more often than they currently do) and Bayesians should be more Bayesian (i.e., be more willing to use informative priors and or make probability statements beyond 95% credible intervals). Or something like that, as I am paraphrasing. That always seemed reasonable. Either way, the dueling--and highly simplified--caricatures of Bayesians vs. Frequentists vs. likelihood folks is largely silly to me. Use the tool that works best for the job at hand, and if you can answer a problem effectively with a well designed experiment and a t-test so be it.