11 ms·
Why isn't everything normally distributed?
- leephillips 9y agoAn example from a book I used to own, called, I think, _Treatment of Experimental Data_: imagine a factory manufacturing ball bearings. They want them all to have the same radius, but because of random errors, the radii will be normally distributed about some mean. If this is true, other random variables, such as the mass, can not be normally distributed.
- prashnts 9y agoSorry, could you explain why mass can't be normally distributed? If, say, the mass of the bearing is related to its radius, then shouldn't it follow similar distribution?
- Sharlin 9y agoMass is proportional to radius cubed. Cubing a normal distribution makes it not normal.
- tgb 9y agoThough it should be noted that "cubing" doesn't refer to just taking the cube, i.e if f(x) = e^{-x^2/2} is proportional to the standard normal distribution then the cube of f(x), namely f * f * f is not the distribution you're considering here. Rather it's the "pushforward measure" [1], in this case is 1/3 y^{-2/3} e^{-y^{2/3}/2} where y=x^3 is the mass and x is the radius. (Up to constants.) [1] https://en.wikipedia.org/wiki/Pushforward_measure https://en.wikipedia.org/wiki/Pushforward_measure
- xyzzyz 9y agoCubing does refer to just taking the cube, but the cube of random variable, and not its probability density function -- the density after cubing is indeed the pushforward.
- bonoboTP 9y agoThis is indeed one confusion that I ran into very often when first learning stats. Transforming the random variable is a very different thing than transforming the probability density function directly. The distinction between a random variable and a distribution (or density) is not made clear enough in classes I think.
- leephillips 9y agoYes, that's why.
- BeetleB 9y agoR = r + e R^{3} = (r+e)^{3} = r^{3} + 3r^{2}e + O(e^{2}) Unless you expect the error to be significant with respect to r, you can ignore the higher order terms. And voila, you get an approximately normal distribution.
- curiousgal 9y agomass = 4/3 pi radius cubed The cube of a normal distribution is indeterminate. Had the relationship between the mass and the radius been linear (M = a * R + b) then yes the mass would have followed a normal distribution as well (with different parameters of course).
- schoen 9y agoThat factor should be 4/3 rather than 3/4. When you differentiate it, you need to get 4πr² for the surface area.
- curiousgal 9y agoYou are correct, my memory failed me and I didn't check. Thank you.
- user5994461 9y agoThat distribution still has the same median, with half the samples on one side and the other half on the other side. That looks a lot like a normal distribution even if it's not one.
- untunedtheramin 9y agoSuppose the radius is normally distributed, and is a random varable X. The mass will be proportional to the volume, which increases with the cube of the radius, so the mass will be a random variable kX^3 (k = 4/3 x pi x density here). X and kX^3 won't have the same distribution.
- escherize 9y agoHmm, why is that?
- wodenokoto 9y agoThis kinda blows my mind. What happens if they do quality control on both mass and radius? I imagine the reason why, in your example it is the radius rather than the mass that is normal, is due to what QA is focused on.
- joshvm 9y agoThe reason that QA is done on radius is because the purpose of a ball bearing is to sit inside its housing to within a fairly tight tolerance. If the balls are out of spec, then potentially the thing the bearing is coupling will be out of alignment/won't rotate freely. The mass is less important. If you did QA on mass, you could easily calculate what mass would be required for a particular radius, but that would let misshapen bearings through. You could do QA on both mass and radius, but unless you have potential contaminants, the radius gives you the mass for free so there isn't much point.
- unabridged 9y agoBut its still log-normal
- cjhanks 9y agoThat's kind of an odd metric to begin with. Radius would probably be derived from the imperfections in the consistency of matter. Whether that distribution is gaussian probably depends on the fabrication process and the materials...
- pishpash 9y agoDepends on whether the radius errors are generated from a separate process than the mass errors. (1) Get a glob of molten metal, mass determined by how much metal is there, some error there; (2) shape it into a ball, and suppose the radius is mainly determined by air that gets trapped, then some nearly independent error there.
- throwaway91111 9y agoThat's true. But the idea that distributions related non-linearly cannot both be normal holds.
- teraflop 9y agoTechnically true, but the smaller the standard deviation of the radius (relative to the mean), the more closely the distribution of masses will approximate a normal distribution.
- andy_wrote 9y agoThis. Technically the radii can't even have a normal distribution, because the support of the normal is (-Inf, Inf) but the radii are bounded below by 0. So you might say that the radii = r + e, where r is constant and e is normal around 0 with variance way, way smaller than r. But then the volume of the ball bearings = K * (r+e)^3, and because r >> e, the largest source of randomness term is going to be 3Kr^2e, which makes the whole thing look pretty normal.
- nerdponx 9y agoFrom the comments: Personally, I don’t find it surprising that not everything is normally distributed. Why should any real phenomenon follow a theoretical limiting distribution anyway, never mind a symmetric, infinite-tailed distribution that is exact only in an unachievable limit? The surprise is that so many things _are_ sufficiently near normality for it to be useful!
- BjoernKW 9y agoBecause many phenomena aren't in fact representations of arbitrary random variables. Take word distribution in any human language for instance. Word frequencies follow a Zipf distribution because it decreases entropy and hence is more efficient.
- oh_sigh 9y agoIt's not arbitrary random variables, it is independent additive variables which leads you to gaussian distributions frequently.
- pmiller2 9y agoThe surprising thing about the CLT is that it applies whenever the means and variances of the summed random variables exist. This is really a very mild condition, but the surprise to me is that the result is independent of the actual distributions of the variables being summed!
- pishpash 9y agoThe convergence rates are different though, so at finite sample sizes and as a practical matter, it does matter.
- ronald_raygun 9y agoInterestingly enough - you don't need random variables to be independent or identically distributed for a CLT to apply. See https://en.wikipedia.org/wiki/Central_limit_theorem#CLT_under_weak_dependence https://en.wikipedia.org/wiki/Central_limit_theorem#CLT_unde... and https://en.wikipedia.org/wiki/Central_limit_theorem#Lyapunov_CLT https://en.wikipedia.org/wiki/Central_limit_theorem#Lyapunov...
- felippee 9y agoBut the variance needs to be bounded. With power laws is is often unbounded. The trick that many get trapped in is that short samples of power laws typically look regular. Estimates of variance will always be finite on finite sample size. Nassim Taleb goes into more detail on this.
- thanatropism 9y agoThen, Taleb is a jerk when someone tries to pin him on one of his wild exaggerations. Even as a quant Taleb was prone to smushing over details to push a narrative. In the 90s already Derman was enabling Taleb into claiming the Black-Scholes formula was an interpolation algorithm already known to option traders and served basically to justify using the risk-free rate as a drift (price trend) parameter in accordance to the "economics establishment". But (as noted by more than one response in the literature) making a different assumption on the drift (or even the distribution) -- i.e. leaving the "Black-Scholes world" and merely interpolating two world-states -- you're left with calibrating a stochastic discount rate that gives put-call parity. But hey, not that technicalities should get in the way of a good story!
- antognini 9y agoAs a (former) astronomer, I've never understood why people assume normal distributions for everything. I understand that there are good theoretical motivations for this --- namely, the normal distribution is the distribution that maximizes entropy for a given mean and variance. But in astronomy, nothing is normally distributed. (At least, nothing comes to mind.) Instead, everything is a power law. The reason for this is that most astrophysical processes are scale free over many orders of magnitude, and if you want a scale free process, it must be distributed as a power law. There's actually a joke in the field that when you get a new dataset, the first thing you do is fit it to a power law. If that doesn't work, you fit it to a broken power law.
- lr4444lr 9y agoI think it has a lot to do with people in the social sciences wishing that the CL Theorem could simplify everything sampled to a single mathematical model they can have tools programmed to do magically when fed the data.
- jtraffic 9y agoIf I hadn't recently read Benoit Mandelbrot's book "The Misbehavior of Markets," I might dismiss this as empty cynicism. But I think there is some truth to it. Macro and financial economists have traditionally been the worst offenders.
- thanatropism 9y agoI'm ignorant of "heavy-tailed statistics" as a field, but the normal distribution in quantitative finance is justified by the Levy characterization of Brownian motion (basically every continuous stochastic process is a drift-diffusion driven by a Brownian) and the Levy representation of a discontinuous process as a sum of a diffusion and a pure jump process. Mind, you might find different distributions when solving diffusion equations (the Cox-Ingersoll-Ross process involves a Bessel distribution if I'm not mistaken). But the diffusion-jump paradigm is much better justified there than distributional (or even finite moment) assumptions in discrete land. Put it this way: if finance is abusing distributional assumptions, there's money being left on the table.
- smcgraw 9y agoSomeone poke Taleb.
- eanzenberg 9y agoWhat's more surprising is how little the central limit theorem holds and is useful, yet is still used all the time to justify poor analysis, usually a/b testing. When the underlying distribution has high variance, as many metrics I've come across with extreme long tailed behavior, the aggregates need large N before they adhere to Gaussian
- jsnk 9y agoWe see Pareto distribution in nature and society a lot. https://en.wikipedia.org/wiki/Pareto_distribution#Applications https://en.wikipedia.org/wiki/Pareto_distribution#Applicatio...
- jtolmar 9y agoI'd take the CLT as everything complicated being normally distributed by default, but with fairly common exceptions: 1 - If the problem isn't actually that complicated, the CLT doesn't do much. 2 - If the problem is dominated by one component, it will still mostly look like that component. 3 - Most ways of slicing a normal distribution lead to other distributions. For example the Rician distribution.
- technofire 9y agoPerhaps a better question is "Why is anything normally distributed?" It appears originally to have been a simplification to make the math more convenient: As Rand Wilcox reports, "Why did Gauss assume that a plot of many observations would be symmetric around some point? Again, the answer does not stem from any empirical argument, but rather a convenient assumption that was in vogue at the time. This assumption can be traced back to the first half of the 18th century and is due to Thomas Simpson. Circa 1755, Thomas Bayes argued that there is no particular reason for assuming symmetry, Simpson recognized and acknowledged the merit of Bayes's argument, but it was unclear how to make any mathematical progress if asymmetry is allowed." (Wilcox, p. 4) Wilcox, R. (2010). Fundamentals of modern statistical methods: Substantially improving power and accuracy (2nd ed.). New York, New York: Springer.[1] [1] http://amzn.to/2tkMRoI http://amzn.to/2tkMRoI
- evanwarfel 9y agoBecause not everything has finite variance.
- abetusk 9y agoAs others have pointed out, power laws are more "normal" than the normal distribution. The reason for this is that if you have the sum independent identically distributed (I.I.D.) random variables (R.V.s), if they converge to a distribution, that distribution is Levy Stable [1], which is power law in it's tails. The Gaussian is a special case in the family of Levy Stable distributions. The article states that "the sum of many independent, additive effects is approximately normally distributed" which is patently false. The sum of many independent random variables with finite variance is normally distributed. Once you relax the finite variance (and in more extreme cases, finite mean) power laws result. There are other ways to generate power laws, including having killed exponential processes [2]. There are many other references that talk about the rediscovery of power laws [3] and give many ways to "naturally" create power laws [3] [4] [5]. The article claims that multiplicative processes lead to log normal distributions. I've heard that this is actually false but unfortunately I don't have enough familiarity to see how this is not true. If anyone has more insight into this I would appreciate a link to an article or other explanation. [1] https://en.wikipedia.org/wiki/Stable_distribution https://en.wikipedia.org/wiki/Stable_distribution [2] http://www.angelfire.com/nv/telka/transfer/powerlaw_expl.pdf http://www.angelfire.com/nv/telka/transfer/powerlaw_expl.pdf [3] https://arxiv.org/pdf/physics/0601192v3.pdf https://arxiv.org/pdf/physics/0601192v3.pdf [4] http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.122.3769&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.122... [5] http://www.angelfire.com/nv/telka/transfer/powerlaw_expl.pdf http://www.angelfire.com/nv/telka/transfer/powerlaw_expl.pdf
- 725686 9y agoNassim Nichalas Taleb doesn't have many nice things about the bell curve in his Black Swan book.
- tictacttoe 9y agoThere are a lot of quantities which are positive definite. If it's bounded from below, it's not Gaussian.
- kgwgk 9y agoStrictly speaking nothing physical is Gaussian, because everything is bounded. There are a finite number of particles in the observable universe.
- graycat 9y agoBecause for random variable X, sometimes need to care about X^2, and when X is normally distributed X^2 is chi-square distributed and not normally distributed. Because under mild assumptions, for arrivals, say, of visitors to a Web site, gas station, hospital, the arrivals form a Poisson process, the times between arrivals are independent and identically exponentially distributed, and that's not normally distributed. Now at nearly any server farm, it is easy to get wide, deep, rapidly flowing oceans of data on the performance of the server farm, and, as U. Grenander once explained to me in his office at Brown, the data is wildly different from what statistics was used to, e.g., in medical data. In that ocean of data, finding anything normally distributed will be very rare. The claims in the OP about many effects are nothing like good evidence for the central limit theorem or normally distributed. E.g., from the renewal theorem, many examples of Poisson processes are from the results of many independent effects. E.g., the usual computer based random number generators return what look like independent, identically distributed random variables uniform on [0,1], and that is not normally distributed. The question in the OP about why not normally distributed is, in one word, just absurd.
- acscott 9y agoFrom the question without even reading a thing, why would any set of events follow a gaussian?
- acscott 9y agoNow, having read some comments, do not get lost in your assumptions (which implies you should know your assumptions). It's really that simple.
- bby 9y agoMy pen is normally distributed.
- clentaminator 9y agoIs the distribution of distributions itself normal?
- vanderZwan 9y ago> Height is influenced by environmental effects as well as genetic effects, such as nutrition, and these environmental effects may be more additive or independent than genetic effects. This makes me wonder if in countries where these environmental effects are mostly optimised (everyone having access to good nutrition), or at least nearly identical for everyone, the normal distribution of height breaks down. Is height normally distributed in the tallest countries in the world? What about the shortest? edit: I'll just copy this question to the comments under his blog, maybe the author has some idea about that. edit2: just noticed the blog post is from 2015... oh well, it was worth a shot.
- ruste 9y agoCan anyone quickly explain to me why some things _are_ normally distributed?
- agentofoblivion 9y agoAll models are wrong, some are useful.
- SophosQ 9y agoI came across this CMU presentation on why the purported ubiquity of power laws must be taken with a grain of salt: https://goo.gl/23PP7v https://goo.gl/23PP7v Check from slide 32. As someone who doesn't have significant experience in statistics, I'd be grateful for an expert's opinion on the arguments presented in this presentation.