4 ms·
> My understanding is that a 95% confidence interval implies that there is a 95% chance the true statistic lies within the interval. What are you saying it mean
by kgwgk 3y ago
> My understanding is that a 95% confidence interval implies that there is a 95% chance the true statistic lies within the interval. What are you saying it means?
If you calculate one thousand confidence intervals in 950 the true statistic will lie within the interval. I'm not sure if that means that there is a 95% chance the true statistic lies within a given interval.
Let me use a extreme example to highlight the issue. Imagine that I use the following procedure to produce a 95% confidence interval for whatever quantity you're interested in:
with 95% probability return the interval [ -1e999 1e999 ]
with 5% probability return the interval [ -1e999 -1e999+1 ]
If you calculate one thousand confidence intervals in 950 the true statistic will lie within the interval. I wouldn't say in any given case that there is a 95% chance the true statistic lies within that particular interval. (There was a 95% chance though.)
This article (open access PDF available) discusses how different methods of producing confidence intervals perform in a toy problem:
The fallacy of placing confidence in confidence intervals
https://link.springer.com/article/10.3758/s13423-015-0947-8 https://link.springer.com/article/10.3758/s13423-015-0947-8
One could argue that those misleading flaws can be avoided by being careful with what confidence intervals do you use and how - but the point is that coverage probabilities are about a set of counterfactual situations and not about the actual situation at hand.
- therobots927 3y agoI explicitely said I'm mostly interested in practical applications, and your example has e999 in it. Thank you for the PDF. I'll take a look at it, but it's really not helping the Bayesian case to start with an example like that. Is the assumption that the interval has a 95% chance of containing the true statistic going to lead to a bridge collapsing or an incorrect financial projection or not?
- kgwgk 3y agoWell, you asked for a clarification about the difference. Of course you're free to not care.