4 ms·

> Often referred to as the cornerstone of statistics Well... often referred to as the central theorem of statistics. Each time you say its name. What's central

by jordigh 12y ago

> Often referred to as the cornerstone of statistics Well... often referred to as the central theorem of statistics. Each time you say its name. What's central is the theorem, not the limit. It was Polya who first called it that, "zentraler Grenzwertsatz". > Why the Central Limit Theorem Works Well... I don't think that's really an explanation at all of why e^(-x^2/2) is such a privileged function. Why would any distribution converge to a normal distribution? It essentially boils down to the Fourier transform. When you take the Fourier transform of the sample means, if you ignore all but the quadratic terms (there are no linear terms if you centralise to mean 0 and variance 1), you get the exponential limit (1 - t^2/2n)^n. That's the Gaussian, which is its own Fourier transform. https://en.wikipedia.org/wiki/Central_limit_theorem#Proof_of_classical_CLT https://en.wikipedia.org/wiki/Central_limit_theorem#Proof_of... In other words, because the Gaussian is its own Fourier transform, sample means converge to the Gaussian.

jamessb 12y ago
> Well... I don't think that's really an explanation at all of why e^(-x^2) is such a privileged function. Why would any distribution converge to a normal distribution? > It essentially boils down to the Fourier transform Precisely. One motivation for taking a fourier transform is that the the p.d.f. of the sum of several random variables is the convolution of corresponding p.d.fs, and that convolution in the original domain is equivalent to multiplication in the fourier domain (by the convolution theorem).
- frozenport 12y ago
  Another is that the integral of rand(x), strictly doesnt converge at the endpoints, although a generalized fourier transform exists.
- shas3 12y ago
  ...and p.d.f's convolve when you derive the distribution of a random variable defined as sum of other independent random variables.
Chinjut 12y ago
It doesn't matter that the Gaussian is its own Fourier transform, as such, does it? It's just that, whatever it is that Fourier transforms into the Gaussian (1 - t^2/2n)^n, that's what sample means converge to. [I suppose it does matter that this is itself Gaussian for establishing the central limit theorem as specifically about Gaussian distributions. I guess my point was that, even if, counterfactually, nothing Fourier transformed into itself, still we would have a central limit theorem about whatever distribution it was that Fourier transformed into the Gaussian]
- TeMPOraL 12y ago
  GP noted why it matters for an actual explanation: > Well... I don't think that's really an explanation at all of why e^(-x^2/2) is such a privileged function. Why would any distribution converge to a normal distribution? The point is, nothing in maths is arbitrary (except axioms); if you see something singled out from the space of many alternatives, there must be a good reason why it's exactly this thing, hidden in the structure of the problem. There's this rule of thumb, also important in programming, that there are generally three amounts: zero, one, or arbitrarily many.
  - Chinjut 12y ago
    I don't think you've caught what I was saying. The GP noted indeed why we should expect the distribution of suitably scaled sample means to converge to the Fourier transform of e^(-x^2/2). I'm saying, it doesn't seem essential to the existence of a CLT type result that this Fourier transform is e^(-x^2/2) itself [i.e., it doesn't seem essential that e^(-x^2/2) happens to be a fixed point of the Fourier transform]; if the Fourier transform of e^(-x^2/2) were some other function, we'd still speak of a Central Limit Theorem and the privileged role of that other function instead. The Fourier transform fixed point phenomenon seems orthogonal to the distribution convergence phenomenon.
    - TeMPOraL 12y ago
      Ok, I guess I didn't catch it. Thanks for the clarification.
mturmon 12y ago
This is a very nice comment, because it points out that the Gaussian "answer" is hidden in the structure of the question. You know the zero'th moment (the distribution has unit mass), and you have removed the first moment (by subtracting the mean), so the next is the second. By passing to a limit, you remove the higher moments from consideration, and isolate on the second-order behavior of the solution near the origin (in characteristic function coordinates). Only the second-order behavior matters, and the Gaussian is the distribution that has a characteristic function with a quadratic behavior at the origin (in the exponent). The real structure of the problem is that it assumes independence, or a similar condition (like mixing) strong enough to mimic independence. Independence is what allows you to pass to the limit and thereby focus on the second-order behavior around the origin. And in practice, you have just swept the bump under the rug, because you now have to justify your independence assumption -- and that's hard. My quibble is with the side comment about the Gaussian being its own Fourier transform. This property is required (i.e., the Gaussian must be a fixed point of the CLT), but the property does not provide any constraint on the class of possible solutions of the CLT. In fact, there is an infinite number of functions that transform into themselves. I can easily construct a new one from any Fourier-transformable function you give me. For example, I can make such a function that looks like a square wave with some wiggles in it. (Reference: http://www.systems.caltech.edu/dsp/ppv/papers/journal08post/PPVIETEeigenFT.pdf http://www.systems.caltech.edu/dsp/ppv/papers/journal08post/...)
- Chinjut 12y ago
  Why is it important for the existence of a CLT type result that the Gaussian be a fixed point of the Fourier transform? Regardless of whether it were or not, we would find that characteristic functions of (suitably scaled) sample means converged to e^(-x^2/2), and thus the Fourier transform of e^(-x^2/2) would be their limiting distribution. That this Fourier transform is e^(-x^2/2) again does not seem essential.
  - mturmon 12y ago
    Hmm, you know, I think you have a point. Clearly any CLT solution must be a fixed point of convolution (as the Gaussian is). The other property ("Gaussian is its own Fourier transform") seemed important, but I'm unable to formalize the argument, and don't think that "fixed point of convolution" requires it. Thanks for keeping me honest. Unless I'm missing something, your observation would render the closing statement in the GP comment ("because the Gaussian is its own Fourier transform") completely irrelevant.
dnautics 12y ago
Note though that e^(-x^2/2) while privileged, is not entirely privileged - because there are other functions that have a central limit property (e.g. 1/(1+x^2)), which is not its own Fourier transform.

by jordigh 12y ago

jamessb 12y ago

> Well... I don't think that's really an explanation at all of why e^(-x^2) is such a privileged function. Why would any distribution converge to a normal distribution? > It essentially boils down to the Fourier transform Precisely. One motivation for taking a fourier transform is that the the p.d.f. of the sum of several random variables is the convolution of corresponding p.d.fs, and that convolution in the original domain is equivalent to multiplication in the fourier domain (by the convolution theorem).

frozenport 12y ago
Another is that the integral of rand(x), strictly doesnt converge at the endpoints, although a generalized fourier transform exists.
shas3 12y ago
...and p.d.f's convolve when you derive the distribution of a random variable defined as sum of other independent random variables.

Chinjut 12y ago

It doesn't matter that the Gaussian is its own Fourier transform, as such, does it? It's just that, whatever it is that Fourier transforms into the Gaussian (1 - t^2/2n)^n, that's what sample means converge to. [I suppose it does matter that this is itself Gaussian for establishing the central limit theorem as specifically about Gaussian distributions. I guess my point was that, even if, counterfactually, nothing Fourier transformed into itself, still we would have a central limit theorem about whatever distribution it was that Fourier transformed into the Gaussian]

TeMPOraL 12y ago
GP noted why it matters for an actual explanation: > Well... I don't think that's really an explanation at all of why e^(-x^2/2) is such a privileged function. Why would any distribution converge to a normal distribution? The point is, nothing in maths is arbitrary (except axioms); if you see something singled out from the space of many alternatives, there must be a good reason why it's exactly this thing, hidden in the structure of the problem. There's this rule of thumb, also important in programming, that there are generally three amounts: zero, one, or arbitrarily many.
- Chinjut 12y ago
  I don't think you've caught what I was saying. The GP noted indeed why we should expect the distribution of suitably scaled sample means to converge to the Fourier transform of e^(-x^2/2). I'm saying, it doesn't seem essential to the existence of a CLT type result that this Fourier transform is e^(-x^2/2) itself [i.e., it doesn't seem essential that e^(-x^2/2) happens to be a fixed point of the Fourier transform]; if the Fourier transform of e^(-x^2/2) were some other function, we'd still speak of a Central Limit Theorem and the privileged role of that other function instead. The Fourier transform fixed point phenomenon seems orthogonal to the distribution convergence phenomenon.
  - TeMPOraL 12y ago
    Ok, I guess I didn't catch it. Thanks for the clarification.

mturmon 12y ago

This is a very nice comment, because it points out that the Gaussian "answer" is hidden in the structure of the question. You know the zero'th moment (the distribution has unit mass), and you have removed the first moment (by subtracting the mean), so the next is the second. By passing to a limit, you remove the higher moments from consideration, and isolate on the second-order behavior of the solution near the origin (in characteristic function coordinates). Only the second-order behavior matters, and the Gaussian is the distribution that has a characteristic function with a quadratic behavior at the origin (in the exponent). The real structure of the problem is that it assumes independence, or a similar condition (like mixing) strong enough to mimic independence. Independence is what allows you to pass to the limit and thereby focus on the second-order behavior around the origin. And in practice, you have just swept the bump under the rug, because you now have to justify your independence assumption -- and that's hard. My quibble is with the side comment about the Gaussian being its own Fourier transform. This property is required (i.e., the Gaussian must be a fixed point of the CLT), but the property does not provide any constraint on the class of possible solutions of the CLT. In fact, there is an infinite number of functions that transform into themselves. I can easily construct a new one from any Fourier-transformable function you give me. For example, I can make such a function that looks like a square wave with some wiggles in it. (Reference: http://www.systems.caltech.edu/dsp/ppv/papers/journal08post/PPVIETEeigenFT.pdf http://www.systems.caltech.edu/dsp/ppv/papers/journal08post/...)

Chinjut 12y ago
Why is it important for the existence of a CLT type result that the Gaussian be a fixed point of the Fourier transform? Regardless of whether it were or not, we would find that characteristic functions of (suitably scaled) sample means converged to e^(-x^2/2), and thus the Fourier transform of e^(-x^2/2) would be their limiting distribution. That this Fourier transform is e^(-x^2/2) again does not seem essential.
- mturmon 12y ago
  Hmm, you know, I think you have a point. Clearly any CLT solution must be a fixed point of convolution (as the Gaussian is). The other property ("Gaussian is its own Fourier transform") seemed important, but I'm unable to formalize the argument, and don't think that "fixed point of convolution" requires it. Thanks for keeping me honest. Unless I'm missing something, your observation would render the closing statement in the GP comment ("because the Gaussian is its own Fourier transform") completely irrelevant.

dnautics 12y ago

Note though that e^(-x^2/2) while privileged, is not entirely privileged - because there are other functions that have a central limit property (e.g. 1/(1+x^2)), which is not its own Fourier transform.