3 ms·

Well, I am not really interested in cookie jars. But I am interested in, for example, particle physics. There we need need simple ways to communicate point esti

Well, I am not really interested in cookie jars. But I am interested in, for example, particle physics. There we need need simple ways to communicate point estimates and the associated uncertainties for various parameters of nature. Intervals are a convenient way to do this. Frequentist confidence intervals have the virtue that they will cover the true parameter at the nominal rate. Bayesian credible intervals in general have no such guarantee. In many cases we can find Bayesian-inspired interval estimators that have good coverage properties. But in some cases there is an irreconcilable conflict. And there you have to choose between long-run correctness and being Bayesian. You might claim that we should do away with intervals and just report posteriors for all physical quantities. This complicates matters slightly without solving the problem. If the true parameters, when ultimately known, consistently end up in very low density regions of the probability distribution (such as far in the tails), we would regard our uncertainty estimates as poor. Again, it is not hard to construct examples where Bayesian methods have poor coverage properties in this sense. (Also, a minor point: Regarding "Do you need to select a strategy up front which gives at least a 70% chance of being right about the jar over iterations where the jar is fixed but the data varies?", one does not need to fix the jar for frequentist methods to have good guarantees. See Wasserman's simulation with the median.) Re: "Given that I drew a cookie with 2 chips on the first draw, what is the chance I draw a cookie with 0 chips on my second draw?", this is not a question about estimating an unknown parameter of a distribution, so it's not statistical in the sense Wasserman is talking about. It's just an elementary probability question that requires knowing something about the jars to answer. Both frequentist and Bayesian statisticians agree on the validity of Bayes rule (and hence how to answer this question once the relevant information is known or assumed); where they differ is on how to conceptualize and estimate unknown parameters of probability distributions.

lalaithion 4y ago
> I am interested in, for example, particle physics. There we need need simple ways to communicate point estimates and the associated uncertainties for various parameters of nature. Okay, but what do you _do_ with a confidence interval once you have it? It's just an abstract object that can't be used to take your knowledge and make better predictions about the future. If I tell you "This new particle decays with a half life of 28 years with a 95% confidence interval of +/- 5 years", can you take that information and use it to estimate the age of an object that started with 236 particles and now has 182 particles? > this is not a question about estimating an unknown parameter of a distribution, so it's not statistical in the sense Wasserman is talking about And a frequentist confidence interval doesn't answer a question about how you should update your knowledge so you can make better predictions in the future, so it's not statistical in the sense bayesians talk about.
- spekcular 4y ago
  The blog post talks about inference, not prediction, so I find it odd you keep bringing up prediction tasks. There are interesting questions and differences here, but it is very much not the subject of the post. A standard frequentist tool for making predictions is the prediction interval. This is the appropriate comparison point for Bayesian prediction methods, and exactly the same issues arise as in the comparison of confidence intervals to credible intervals (or posteriors). Namely, frequentist prediction intervals have guaranteed error control, while Bayesian predictions generally do not. So in certain cases you have to choose between being right most of time about your predictions, and being Bayesian.

lalaithion 4y ago

> I am interested in, for example, particle physics. There we need need simple ways to communicate point estimates and the associated uncertainties for various parameters of nature. Okay, but what do you _do_ with a confidence interval once you have it? It's just an abstract object that can't be used to take your knowledge and make better predictions about the future. If I tell you "This new particle decays with a half life of 28 years with a 95% confidence interval of +/- 5 years", can you take that information and use it to estimate the age of an object that started with 236 particles and now has 182 particles? > this is not a question about estimating an unknown parameter of a distribution, so it's not statistical in the sense Wasserman is talking about And a frequentist confidence interval doesn't answer a question about how you should update your knowledge so you can make better predictions in the future, so it's not statistical in the sense bayesians talk about.

spekcular 4y ago
The blog post talks about inference, not prediction, so I find it odd you keep bringing up prediction tasks. There are interesting questions and differences here, but it is very much not the subject of the post. A standard frequentist tool for making predictions is the prediction interval. This is the appropriate comparison point for Bayesian prediction methods, and exactly the same issues arise as in the comparison of confidence intervals to credible intervals (or posteriors). Namely, frequentist prediction intervals have guaranteed error control, while Bayesian predictions generally do not. So in certain cases you have to choose between being right most of time about your predictions, and being Bayesian.