4 ms·

I'm having trouble understanding your example. What value is the 93.75% confidence interval for? Is it for the population mean? If so, why does the sample order

I'm having trouble understanding your example. What value is the 93.75% confidence interval for? Is it for the population mean? If so, why does the sample order influence the result?

6gvONxR4sf7o 6y ago
An x% confidence interval takes data D and produces an interval i. For some data generating process from parameter m to a random dataset D (random like if you do the same experiment again, you’ll get different data), the probability that i(D) contains m is x%. That’s the definition of an x% CI. In my example i(D) is a function of the data (a function of the ordering), and D is a random dataset. Since it’s iid by assumption (sneakily also assuming the probability of an exact repeat is zero), the probability that the interval contains all numbers is 93.75% (1-1/2^4). Otherwise it’s the empty set. Unpacking that, suppose you have a real number m. The probability that i(D) will contain m (with D as the random variable) is 93.75%, so it is a valid confidence interval for m. m could be the population mean, the population median, your dog’s age, whatever. The interval depends on the data, but not on the parameter, and the definition of a CI says that’s fine. It’s a demonstration that definition of a CI alone isn’t really useful for reasoning about a parameter given an interval. You need to know more about the specific data generating process and function i that led to it in order to make sure it’s useful.
- zmmmmm 6y ago
  I'm confused by your example also ... if the points are iid then how can the order influence the estimate? They are independent of each other, so there is no way for points 1 - 5 to influence the next data value sampled from the distribution. Or to put it another way, if i(D) is a function of the ordering, then isn't by definition the ultimate random process observed through i(D) not iid even if D is iid?
  - 6gvONxR4sf7o 6y ago
    If the points are iid, it’s crazy for the order to affect the estimate, but not off limits. This paper goes into it more thoroughly: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.691.5256&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.691...
    - zmmmmm 6y ago
      thanks - that was an amazing read!
- tpoacher 6y ago
  that is not a correct interpretation of a confidence interval. what you describe refers to a posterior probability, which is not what CIs do. See Greenland et al for a good paper on this.
  - 6gvONxR4sf7o 6y ago
    It is most definitely not a posterior probability, but it’s always hard to tersely write out which process you’re describing in plain English and no formalism. All the probabilities I mention are probabilities in the sense of a CI. And I’m too lazy to write it out thoroughly. This paper describes the situation more thoroughly http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.691.5256&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.691...
    - tpoacher 6y ago
      My main objection was regarding the iffy sentence that "the probability that i(D) contains m is x%", as opposed to, say, you would expect "x% of those randomly generated intervals to contain m". But, fair enough, I appreciate your sentiment about 'terseness'. One can only be so nitpicky with words when trying to communicate, before starting to sound like a criminal defense lawyer ...

I'm having trouble understanding your example. What value is the 93.75% confidence interval for? Is it for the population mean? If so, why does the sample order influence the result?

6gvONxR4sf7o 6y ago

An x% confidence interval takes data D and produces an interval i. For some data generating process from parameter m to a random dataset D (random like if you do the same experiment again, you’ll get different data), the probability that i(D) contains m is x%. That’s the definition of an x% CI. In my example i(D) is a function of the data (a function of the ordering), and D is a random dataset. Since it’s iid by assumption (sneakily also assuming the probability of an exact repeat is zero), the probability that the interval contains all numbers is 93.75% (1-1/2^4). Otherwise it’s the empty set. Unpacking that, suppose you have a real number m. The probability that i(D) will contain m (with D as the random variable) is 93.75%, so it is a valid confidence interval for m. m could be the population mean, the population median, your dog’s age, whatever. The interval depends on the data, but not on the parameter, and the definition of a CI says that’s fine. It’s a demonstration that definition of a CI alone isn’t really useful for reasoning about a parameter given an interval. You need to know more about the specific data generating process and function i that led to it in order to make sure it’s useful.

zmmmmm 6y ago
I'm confused by your example also ... if the points are iid then how can the order influence the estimate? They are independent of each other, so there is no way for points 1 - 5 to influence the next data value sampled from the distribution. Or to put it another way, if i(D) is a function of the ordering, then isn't by definition the ultimate random process observed through i(D) not iid even if D is iid?
- 6gvONxR4sf7o 6y ago
  If the points are iid, it’s crazy for the order to affect the estimate, but not off limits. This paper goes into it more thoroughly: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.691.5256&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.691...
  - zmmmmm 6y ago
    thanks - that was an amazing read!
tpoacher 6y ago
that is not a correct interpretation of a confidence interval. what you describe refers to a posterior probability, which is not what CIs do. See Greenland et al for a good paper on this.
- 6gvONxR4sf7o 6y ago
  It is most definitely not a posterior probability, but it’s always hard to tersely write out which process you’re describing in plain English and no formalism. All the probabilities I mention are probabilities in the sense of a CI. And I’m too lazy to write it out thoroughly. This paper describes the situation more thoroughly http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.691.5256&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.691...
  - tpoacher 6y ago
    My main objection was regarding the iffy sentence that "the probability that i(D) contains m is x%", as opposed to, say, you would expect "x% of those randomly generated intervals to contain m". But, fair enough, I appreciate your sentiment about 'terseness'. One can only be so nitpicky with words when trying to communicate, before starting to sound like a criminal defense lawyer ...