4 ms·

The Normal Distribution: A derivation from basic principles [pdf]

28 points by nkron 12y ago

csense 12y ago
Thanks for posting this. I've been looking for intuitions about what's so special about the normal distribution.
- widdma 12y ago
  While this is a nice little derivation, the reason the normal distribution is important (and common) is the Central limit theorem[1]. This loosely means that if you have n samples that are independently drawn from the same distribution, their sum approaches a normal distribution for large n. The generality comes from the fact that this happens (almost) independently of distribution the samples come from. [1]https://en.wikipedia.org/wiki/Central_limit_theorem https://en.wikipedia.org/wiki/Central_limit_theorem
  - stiff 12y ago
    This is often given as the reason in the textbooks, but it actually is a sloppy argument, for there is no reason to presuppose sums of independent variables (they do not have to be drawn from the same distribution in some variants of the CLT) are so ubiquitous in nature. A more convincing reason is given by: http://en.wikipedia.org/wiki/Maximum_entropy_probability_distribution http://en.wikipedia.org/wiki/Maximum_entropy_probability_dis...
    - mturmon 12y ago
      Well, I think textbooks often give the maxent justification as well, but probably neither is strictly superior to the other. Often, where a measurement is perturbed by a litany of physically unlinked disturbances (e.g., antenna noise), the CLT argument is compelling. In a case like antenna or front-end electronics noise, justifying the maxent argument would require buying in to an almost mystical belief by comparison. Interestingly, the two seem to have a deep connection related to thermodynamics, e.g. http://en.wikipedia.org/wiki/Thermal_fluctuations http://en.wikipedia.org/wiki/Thermal_fluctuations, where you can see both as expansions of a configuration count up to second order in the exponent. This is manifest in the detectable remnants of the Big Bang, which are a particular realization of a Gaussian random field that was present due to quantum fluctuations of a primordial system. It's funny to think of the universe as a draw from a random number generator.
- deleted 12y ago
  [deleted]
quarterwave 12y ago
Isn't the key point here about isotropy, similar to the velocities in the ideal gas?
- mturmon 12y ago
  I think you're right, but it's really isotropy in combination with independence of x and y. Isotropy means you really are dealing with a function of r where r^2 = x^2 + y^2, and independence means the resulting function p(x,y) must factor as p(x) * p(y). This then forces the exponential form.

28 points by nkron 12y ago

csense 12y ago

Thanks for posting this. I've been looking for intuitions about what's so special about the normal distribution.

widdma 12y ago
While this is a nice little derivation, the reason the normal distribution is important (and common) is the Central limit theorem[1]. This loosely means that if you have n samples that are independently drawn from the same distribution, their sum approaches a normal distribution for large n. The generality comes from the fact that this happens (almost) independently of distribution the samples come from. [1]https://en.wikipedia.org/wiki/Central_limit_theorem https://en.wikipedia.org/wiki/Central_limit_theorem
- stiff 12y ago
  This is often given as the reason in the textbooks, but it actually is a sloppy argument, for there is no reason to presuppose sums of independent variables (they do not have to be drawn from the same distribution in some variants of the CLT) are so ubiquitous in nature. A more convincing reason is given by: http://en.wikipedia.org/wiki/Maximum_entropy_probability_distribution http://en.wikipedia.org/wiki/Maximum_entropy_probability_dis...
  - mturmon 12y ago
    Well, I think textbooks often give the maxent justification as well, but probably neither is strictly superior to the other. Often, where a measurement is perturbed by a litany of physically unlinked disturbances (e.g., antenna noise), the CLT argument is compelling. In a case like antenna or front-end electronics noise, justifying the maxent argument would require buying in to an almost mystical belief by comparison. Interestingly, the two seem to have a deep connection related to thermodynamics, e.g. http://en.wikipedia.org/wiki/Thermal_fluctuations http://en.wikipedia.org/wiki/Thermal_fluctuations, where you can see both as expansions of a configuration count up to second order in the exponent. This is manifest in the detectable remnants of the Big Bang, which are a particular realization of a Gaussian random field that was present due to quantum fluctuations of a primordial system. It's funny to think of the universe as a draw from a random number generator.
deleted 12y ago
[deleted]

quarterwave 12y ago

Isn't the key point here about isotropy, similar to the velocities in the ideal gas?

mturmon 12y ago
I think you're right, but it's really isotropy in combination with independence of x and y. Isotropy means you really are dealing with a function of r where r^2 = x^2 + y^2, and independence means the resulting function p(x,y) must factor as p(x) * p(y). This then forces the exponential form.