3 ms·
Confidence intervals are weird because of their very minimal definition. My favorite confidence interval procedure for iid data demonstrates why you need a tigh
by 6gvONxR4sf7o 6y ago
Confidence intervals are weird because of their very minimal definition. My favorite confidence interval procedure for iid data demonstrates why you need a tighter definition for a useful interval.
For a 93.75% confidence interval, draw 5 points (iid). If the last four are all greater than the first one, your CI is the whole real number line, otherwise it’s the empty set.
Once you draw some actual data and get a specific interval, you want to ask about some degree of belief that your specific interval contains the actual parameter. In the case that your CI is all numbers, you know for a fact that it contains the true parameter value. In the case that your CI is the empty set, you know for a fact that it doesn’t contain the true parameter value.
I like this CI procedure because it demonstrates two things. 1) The kind of reasoning for going forward from an unknown parameter to a random interval is very different than what you have working backwards from a specific interval back to the parameter. That asymmetry can be WEIRD. 2) The weirdness is possible if you limit yourself to only the CI definition, meaning that if you want it to be useful, you need something that rules out weird shit like my example.
The properties of specific CI procedures people actually use are generally much much better than what is allowed by the definition of a CI. If you want useful reasoning backwards from the interval, don’t try to reason solely from the definition of a CI.
- hexane360 6y agoI'm having trouble understanding your example. What value is the 93.75% confidence interval for? Is it for the population mean? If so, why does the sample order influence the result?
- 6gvONxR4sf7o 6y agoAn x% confidence interval takes data D and produces an interval i. For some data generating process from parameter m to a random dataset D (random like if you do the same experiment again, you’ll get different data), the probability that i(D) contains m is x%. That’s the definition of an x% CI. In my example i(D) is a function of the data (a function of the ordering), and D is a random dataset. Since it’s iid by assumption (sneakily also assuming the probability of an exact repeat is zero), the probability that the interval contains all numbers is 93.75% (1-1/2^4). Otherwise it’s the empty set. Unpacking that, suppose you have a real number m. The probability that i(D) will contain m (with D as the random variable) is 93.75%, so it is a valid confidence interval for m. m could be the population mean, the population median, your dog’s age, whatever. The interval depends on the data, but not on the parameter, and the definition of a CI says that’s fine. It’s a demonstration that definition of a CI alone isn’t really useful for reasoning about a parameter given an interval. You need to know more about the specific data generating process and function i that led to it in order to make sure it’s useful.
- zmmmmm 6y agoI'm confused by your example also ... if the points are iid then how can the order influence the estimate? They are independent of each other, so there is no way for points 1 - 5 to influence the next data value sampled from the distribution. Or to put it another way, if i(D) is a function of the ordering, then isn't by definition the ultimate random process observed through i(D) not iid even if D is iid?
- 6gvONxR4sf7o 6y agoIf the points are iid, it’s crazy for the order to affect the estimate, but not off limits. This paper goes into it more thoroughly: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.691.5256&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.691...
- zmmmmm 6y agothanks - that was an amazing read!
- tpoacher 6y agothat is not a correct interpretation of a confidence interval. what you describe refers to a posterior probability, which is not what CIs do. See Greenland et al for a good paper on this.
- 6gvONxR4sf7o 6y agoIt is most definitely not a posterior probability, but it’s always hard to tersely write out which process you’re describing in plain English and no formalism. All the probabilities I mention are probabilities in the sense of a CI. And I’m too lazy to write it out thoroughly. This paper describes the situation more thoroughly http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.691.5256&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.691...
- 6y ago