4 ms·

This comes at a good time! I was just talking with my spouse about why you put n-1 in the denominator when you calculate variance. I always liked "because you

This comes at a good time! I was just talking with my spouse about why you put n-1 in the denominator when you calculate variance. I always liked "because you used up a degree of freedom calculating the mean," but I feel like that's kind of hand-wavy. (Obviously, I'm not a statistician.)

clircle 7y ago
You can argue either way. dividing by n is the maximum likelihood estimator, but dividing by n-1 is the unbiased estimator. Depends on what you like. Most people prefer unbiased estimators when they are available.
- benrbray 7y ago
  A mathematician once gave me a convincing argument for dividing by n+1 instead, but I have since forgotten his reason.
  - mturmon 7y ago
    The argument is that this factor can minimize the mean squared error of the variance estimate, at the cost of some bias. In general, the small correction to the lead factor depends on the fourth moment. For a Gaussian, you get 1/(n+1). This is an example of a “shrinkage estimator”, which comes up a lot - introduce some bias but get a smaller MSE. For more, see: https://en.wikipedia.org/wiki/Bessel%27s_correction https://en.wikipedia.org/wiki/Bessel%27s_correction
- sanderjd 7y ago
  What does it mean for it to be "unbiased"? What does it mean to "use up" a degree of freedom? I don't mind if things just can't really be explained intuitively because they are fundamentally technical, but your explanation and the parent's both do this thing where it sounds like it's explaining things in plain common language, but isn't actually because it isn't clear what those plain words mean in this context.
  - mehrdadn 7y ago
    "Unbiased" means the error is on average zero.
  - absherwin 7y ago
    Unbiased means that if I draw infinitely many random samples from a population and average a statistic (in this case standard deviation) across all the samples, the answer will be the statistic computed from the population itself. If one divides by n instead of n-1, the estimate for standard deviation will be be (n-1)/n too small. One reading this might think, "Wait! We're going to infinity so the ratio converges to 1." That's true if the size of each sample also goes to infinity but not if we draw millions of ten item samples. As for using up a degree of freedom, the easiest way to build intuition for why this is a useful concept is to think about very small samples. Let's say I draw a sample of 1 item. By definition the item is equal to the mean so I receive no information about the standard deviation. Conversely, if someone had told me the mean in advance, I could learn a bit about the standard deviation with a single sample. This carries on beyond one in diminishing amounts. Imagine I draw two items. There's some probability that they're both on the same side of the mean, in that case, I'll estimate my sample mean as being between those number and underestimate the standard deviation. Note that I'd still underestimate it even with the bias correction, it's just that that factor compensates just enough that it balances out over all cases. A simple, concrete way to convince yourself that this is real is to consider the standard deviation of a variable that has an equal probability of being 1 or 0. The standard deviation is 0.5. But if we randomly sample two items, 50% of the time they'll be the same and we'll estimate the standard deviation as zero. The other 50% of the time, we'll get the right answer. Hence, our average is half the right answer (n/(n-1)=2/1). The correction makes the standard deviation double what it should be half the same while remaining zero in the other cases. This also suggests why dividing by n is referred to as a the maximum likelihood estimator.
    - sanderjd 7y ago
      Thanks! This is helpful!
    - krychu 7y ago
- adouzzy 7y ago
  Asymptotically, all the same. If sample size is small, it's a bad estimator anyway.
cjhveal 7y ago
For those wondering, this is what the resource at hand says about the matter: "To be precise, when this mean is calculated, the sum of the squared deviations is divided by one less than the sample size rather than the sample size itself. There's no reason why it must be done this way, but this is the modern convention. It's not important that this seem the most natural measure of spread. It's the way it's done. You can just accept it (which I recommend) or you'll have to study the mathematics behind it. But, that's another course."
pknerd 7y ago
You guys might like to watch this video: https://www.youtube.com/watch?v=dq_D30kyR1A https://www.youtube.com/watch?v=dq_D30kyR1A https://www.youtube.com/watch?v=ANsVodOu1Tg https://www.youtube.com/watch?v=ANsVodOu1Tg
mikorym 7y ago
Could you link to which of the pages in OP's book (webpage) this is? I tried in vain to go to the likely one and still couldn't find where the variance gets introduced!

clircle 7y ago

You can argue either way. dividing by n is the maximum likelihood estimator, but dividing by n-1 is the unbiased estimator. Depends on what you like. Most people prefer unbiased estimators when they are available.

benrbray 7y ago
A mathematician once gave me a convincing argument for dividing by n+1 instead, but I have since forgotten his reason.
- mturmon 7y ago
  The argument is that this factor can minimize the mean squared error of the variance estimate, at the cost of some bias. In general, the small correction to the lead factor depends on the fourth moment. For a Gaussian, you get 1/(n+1). This is an example of a “shrinkage estimator”, which comes up a lot - introduce some bias but get a smaller MSE. For more, see: https://en.wikipedia.org/wiki/Bessel%27s_correction https://en.wikipedia.org/wiki/Bessel%27s_correction
sanderjd 7y ago
What does it mean for it to be "unbiased"? What does it mean to "use up" a degree of freedom? I don't mind if things just can't really be explained intuitively because they are fundamentally technical, but your explanation and the parent's both do this thing where it sounds like it's explaining things in plain common language, but isn't actually because it isn't clear what those plain words mean in this context.
- mehrdadn 7y ago
  "Unbiased" means the error is on average zero.
- absherwin 7y ago
  Unbiased means that if I draw infinitely many random samples from a population and average a statistic (in this case standard deviation) across all the samples, the answer will be the statistic computed from the population itself. If one divides by n instead of n-1, the estimate for standard deviation will be be (n-1)/n too small. One reading this might think, "Wait! We're going to infinity so the ratio converges to 1." That's true if the size of each sample also goes to infinity but not if we draw millions of ten item samples. As for using up a degree of freedom, the easiest way to build intuition for why this is a useful concept is to think about very small samples. Let's say I draw a sample of 1 item. By definition the item is equal to the mean so I receive no information about the standard deviation. Conversely, if someone had told me the mean in advance, I could learn a bit about the standard deviation with a single sample. This carries on beyond one in diminishing amounts. Imagine I draw two items. There's some probability that they're both on the same side of the mean, in that case, I'll estimate my sample mean as being between those number and underestimate the standard deviation. Note that I'd still underestimate it even with the bias correction, it's just that that factor compensates just enough that it balances out over all cases. A simple, concrete way to convince yourself that this is real is to consider the standard deviation of a variable that has an equal probability of being 1 or 0. The standard deviation is 0.5. But if we randomly sample two items, 50% of the time they'll be the same and we'll estimate the standard deviation as zero. The other 50% of the time, we'll get the right answer. Hence, our average is half the right answer (n/(n-1)=2/1). The correction makes the standard deviation double what it should be half the same while remaining zero in the other cases. This also suggests why dividing by n is referred to as a the maximum likelihood estimator.
  - sanderjd 7y ago
    Thanks! This is helpful!
  - krychu 7y ago
adouzzy 7y ago
Asymptotically, all the same. If sample size is small, it's a bad estimator anyway.

cjhveal 7y ago

For those wondering, this is what the resource at hand says about the matter: "To be precise, when this mean is calculated, the sum of the squared deviations is divided by one less than the sample size rather than the sample size itself. There's no reason why it must be done this way, but this is the modern convention. It's not important that this seem the most natural measure of spread. It's the way it's done. You can just accept it (which I recommend) or you'll have to study the mathematics behind it. But, that's another course."

pknerd 7y ago

You guys might like to watch this video: https://www.youtube.com/watch?v=dq_D30kyR1A https://www.youtube.com/watch?v=dq_D30kyR1A https://www.youtube.com/watch?v=ANsVodOu1Tg https://www.youtube.com/watch?v=ANsVodOu1Tg

mikorym 7y ago

Could you link to which of the pages in OP's book (webpage) this is? I tried in vain to go to the likely one and still couldn't find where the variance gets introduced!