4 ms·

My (mis-?) understanding of the Central Limit Theorem is that the average of >30 measurements increases the accuracy. https://en.wikipedia.org/wiki/Central_limi

My (mis-?) understanding of the Central Limit Theorem is that the average of >30 measurements increases the accuracy. https://en.wikipedia.org/wiki/Central_limit_theorem https://en.wikipedia.org/wiki/Central_limit_theorem

moron4hire 3y ago
No, that's increasing the precision, not accuracy. Many imprecise measurements can be averaged to a more precise measurement. Say we had a perfect ruler to measure the length of something. It's absolutely precise and accurate. All measurements from it return the exact, same value, and preternaturally we know it's the "correct" value. Now say some bandit comes in while we're not looking and adds a small chunck of diamond to the end of the ruler without telling us. Our ruler is still precise, but no longer accurate. If we take many measurements with it, they always come back with the same value. Averaging those values does not improve the accuracy at all. Alternatively, say the bandit starts randomly changing the temperature of the room we are in. Thermal expansion is constantly changing the length of the ruler. The average length of the ruler is still the same, so it's still accurate, but it's no longer precise. We could average the measurements we take with it and get a more precise measurement. What the central limit theorem says is that the averages of our measurements will be normally distributed, even if the change in temperature is not.
- kelseyfrog 3y ago
  This really doesn't make much sense in the context of the measurement which is bounding the value(0) with 90% confidence[1]. 1. https://arxiv.org/pdf/2212.11841.pdf https://arxiv.org/pdf/2212.11841.pdf
  - random314 3y ago
    I think op is talking about bias vs variance. In some cases we have a bias variance trade off.
    - kelseyfrog 3y ago
      Yep, and I'm talking about hypothesis testing which is what's being done in the paper.
      - random314 3y ago
        Hypothesis testing cannot rule out systemic bias errors present in both control and treatment populations.
        kelseyfrog 3y ago
        Are you pontificating about systemic bias or is this a specific critique of the techniques used to simultaneously detect the upper and lower doublets in a single shot of the experiment?
        random314 3y ago
        I am trying to explain to you what moron4hire is talking about. Hypothesis testing doesn't magically fix the issue he is discussing, as hypothesis testing works by collecting many imprecise measurements to average it out to a more precise level.
- klausnrooster 3y ago
  That's all true. I am biased because I am responsible for keeping hundreds of devices calibrated, and keeping a lab within 2C of 23C. The CLT implies that the sample mean is more likely to be close to the true population mean as the sample size increases. Not the same as more accurate. Shame on me!
  - random314 3y ago
    The number of people in this thread not understanding the difference between bias and variance errors is too much. http://scott.fortmann-roe.com/docs/BiasVariance.html http://scott.fortmann-roe.com/docs/BiasVariance.html

moron4hire 3y ago

No, that's increasing the precision, not accuracy. Many imprecise measurements can be averaged to a more precise measurement. Say we had a perfect ruler to measure the length of something. It's absolutely precise and accurate. All measurements from it return the exact, same value, and preternaturally we know it's the "correct" value. Now say some bandit comes in while we're not looking and adds a small chunck of diamond to the end of the ruler without telling us. Our ruler is still precise, but no longer accurate. If we take many measurements with it, they always come back with the same value. Averaging those values does not improve the accuracy at all. Alternatively, say the bandit starts randomly changing the temperature of the room we are in. Thermal expansion is constantly changing the length of the ruler. The average length of the ruler is still the same, so it's still accurate, but it's no longer precise. We could average the measurements we take with it and get a more precise measurement. What the central limit theorem says is that the averages of our measurements will be normally distributed, even if the change in temperature is not.

kelseyfrog 3y ago
This really doesn't make much sense in the context of the measurement which is bounding the value(0) with 90% confidence[1]. 1. https://arxiv.org/pdf/2212.11841.pdf https://arxiv.org/pdf/2212.11841.pdf
- random314 3y ago
  I think op is talking about bias vs variance. In some cases we have a bias variance trade off.
  - kelseyfrog 3y ago
    Yep, and I'm talking about hypothesis testing which is what's being done in the paper.
    - random314 3y ago
      Hypothesis testing cannot rule out systemic bias errors present in both control and treatment populations.
      - kelseyfrog 3y ago
        Are you pontificating about systemic bias or is this a specific critique of the techniques used to simultaneously detect the upper and lower doublets in a single shot of the experiment?
        random314 3y ago
        I am trying to explain to you what moron4hire is talking about. Hypothesis testing doesn't magically fix the issue he is discussing, as hypothesis testing works by collecting many imprecise measurements to average it out to a more precise level.
klausnrooster 3y ago
That's all true. I am biased because I am responsible for keeping hundreds of devices calibrated, and keeping a lab within 2C of 23C. The CLT implies that the sample mean is more likely to be close to the true population mean as the sample size increases. Not the same as more accurate. Shame on me!
- random314 3y ago
  The number of people in this thread not understanding the difference between bias and variance errors is too much. http://scott.fortmann-roe.com/docs/BiasVariance.html http://scott.fortmann-roe.com/docs/BiasVariance.html