3 ms·

CLT doesn't say that distributions become Gaussian, it says that if you take sets of samples from a distribution, the distribution of means of those sets of sam

CLT doesn't say that distributions become Gaussian, it says that if you take sets of samples from a distribution, the distribution of means of those sets of samples is often Gaussian (under certain assumptions) even if the underlying distribution is not Gaussian. It's incredible how often people get this wrong. The distribution of means of samples is not always Gaussian (eg CLT doesn't hold if the underlying is a Cauchy distribution for example) and the underlying distribution won't magically "become Gaussian" even if you wait a long time unless it was Gaussian to start with.

tomtom1337 3y ago
Your first paragraph is how I had understood the CLT as well. I’m wondering now if the author of the post knows more than me and is correct about what he’s talking about, or if he’s wrong.
- kqr 3y ago
  If the mean is Gaussian then so is the sum – one is simply the other scaled by a non-random amount. More formally, if we have x ~ N(m, s) then nx ~ N(nm, sqrt(n)s).
  - seanhunter 3y ago
    The sum of a linear combination of Gaussians is Gaussian but that's not what the CLT is talking about. It is saying that the distribution of _sample means_ taken from some distribution is Gaussian. The distribution of means of samples talked about by the CLT is quite different from the underlynig distribution. Its variance is \sigma^2/n if the original distribution has variance of \sigma^2 for example. It does have the same mean as the underlying distribution however. Edit to add: Here's a small python program so you can run it in a jupyter notebook or something and see for yourself: import numpy as np import matplotlib.pyplot as plt import seaborn as sns broken_dice_probabilities = [int(x!=3)*0.2 for x in range(6)] broken_dice_sides = [x+1 for x in range(6)] n=1000000 population = np.random.choice(broken_dice_sides, replace=True, p=broken_dice_probabilities, size=n) ax=sns.histplot(population) ax.set_title("Underlying population") plt.show() n_samples = 100000 sample_size = 20 clt_samples=np.zeros(n_samples) for i in range(n_samples): this_sample = np.random.choice(population, replace=True, size=sample_size) clt_samples[i]=np.mean(this_sample) ax=sns.histplot(clt_samples) ax.set_title("Sample means") plt.show() I make a very obviously non-Gaussian distribution - a broken 6-sided die that cannot roll a four. You see the underlying distribution is uniform except one number where p=0, then I take samples from this distribution and plot their mean. Seeing is believing, the distribution of these is Gaussian. Now, note that you can keep sampling for infinity from the underlying distribution - the results will never be Gaussian.
    - kqr 3y ago
      If the distribution of sample means tends to Gaussian, then the distribution of sample sums tend to the same Gaussian (except scaled.) So if the original X_i are drawn from any distribution with defined with mean m and variance s², then the CLT says that in the limit, the sample mean is distributed N(m, s), and, equivalently, the sample sum is distributed N(nm, sqrt(n)s). The question is when the "in the limit" results starts being practically useful, and that depends on the distribution of the original X_i.
      - seanhunter 3y ago
        Yes. Sorry I misunderstood your point.

tomtom1337 3y ago

Your first paragraph is how I had understood the CLT as well. I’m wondering now if the author of the post knows more than me and is correct about what he’s talking about, or if he’s wrong.

kqr 3y ago
If the mean is Gaussian then so is the sum – one is simply the other scaled by a non-random amount. More formally, if we have x ~ N(m, s) then nx ~ N(nm, sqrt(n)s).
- seanhunter 3y ago
  The sum of a linear combination of Gaussians is Gaussian but that's not what the CLT is talking about. It is saying that the distribution of _sample means_ taken from some distribution is Gaussian. The distribution of means of samples talked about by the CLT is quite different from the underlynig distribution. Its variance is \sigma^2/n if the original distribution has variance of \sigma^2 for example. It does have the same mean as the underlying distribution however. Edit to add: Here's a small python program so you can run it in a jupyter notebook or something and see for yourself: import numpy as np import matplotlib.pyplot as plt import seaborn as sns broken_dice_probabilities = [int(x!=3)*0.2 for x in range(6)] broken_dice_sides = [x+1 for x in range(6)] n=1000000 population = np.random.choice(broken_dice_sides, replace=True, p=broken_dice_probabilities, size=n) ax=sns.histplot(population) ax.set_title("Underlying population") plt.show() n_samples = 100000 sample_size = 20 clt_samples=np.zeros(n_samples) for i in range(n_samples): this_sample = np.random.choice(population, replace=True, size=sample_size) clt_samples[i]=np.mean(this_sample) ax=sns.histplot(clt_samples) ax.set_title("Sample means") plt.show() I make a very obviously non-Gaussian distribution - a broken 6-sided die that cannot roll a four. You see the underlying distribution is uniform except one number where p=0, then I take samples from this distribution and plot their mean. Seeing is believing, the distribution of these is Gaussian. Now, note that you can keep sampling for infinity from the underlying distribution - the results will never be Gaussian.
  - kqr 3y ago
    If the distribution of sample means tends to Gaussian, then the distribution of sample sums tend to the same Gaussian (except scaled.) So if the original X_i are drawn from any distribution with defined with mean m and variance s², then the CLT says that in the limit, the sample mean is distributed N(m, s), and, equivalently, the sample sum is distributed N(nm, sqrt(n)s). The question is when the "in the limit" results starts being practically useful, and that depends on the distribution of the original X_i.
    - seanhunter 3y ago
      Yes. Sorry I misunderstood your point.