3 ms·

Do all probability distributions have known processes that generate them? Like how you can make a normal distribution with a Galton box? Is there something like

Do all probability distributions have known processes that generate them? Like how you can make a normal distribution with a Galton box? Is there something like that with power law distributions? It seems like the deal with power laws is that the events aren't independent (an event becomes more likely the more it occurs, or the rich get richer).

antognini 10y ago
My background is in astronomy, where power laws are ubiquitous. (There's a joke that when you get some data, you fit it to a power law. If that's not a good enough fit, you fit it to a broken power law.) The reason they're so common is that a power law is generally the distribution you get anytime you have a scale-invariant process.
klodolph 10y ago
A Galton box generates a binomial distribution. Most probability distributions are designed to model some kind of process. The time between clicks of a geiger counter is exponential, the number of clicks in a second is poisson. The amount of time your geiger counter lasts before it breaks and you have to buy a new one is weibull. That's the whole reason why we invented probability distributions in the first place: to model processes.
- dekhn 10y ago
  The Galton box generates a binomial distribution which when taken to the limit of n->infinity, produces a normal distribution.
  - klodolph 10y ago
    This is incorrect, you need to take a limit as n->infinity while dividing by a factor of √n, otherwise the limit does not exist. But Galton boxes in practice are not very large, so they aren't very good approximations of normal distributions.
AlexCoventry 10y ago
What do you mean by all probability distributions? Any measurable function on a measurable space with Lebesgue integral 1 can be considered a probability distribution, so the answer to the technical interpretation of your question is no.
- thomasahle 10y ago
  Well, they do all have processes that approximately generate them, as by Monte Carlo sampling.
abetusk 10y ago
Power laws come up because they're the limiting distribution for a (common) class of processes. You can generate power law tailed distributions easily without dependence. The class of distributions that both power law tails and the Normal distribution fall under are called "Levy-stable distributions" [1]. Given the sum of independent, identically distributed (I.I.D.) random variables that converge to a stable distribution, the limiting distribution is Levy-stable (read, power law tails). If you the sum of the I.I.D. random variables has finite variance, the Gaussian distribution is the limiting distribution, which is a special case of the Levy-stable distributions. If each of the I.I.D. random variables has infinite second moment (e.g. infinite variance) and converges, then it converges to something with power law tails. We've been spoiled to think that the Gaussian's are the only or most common limiting distribution (thus the name "Normal") but Levy-stable distributions are another class of "common" distributions for the same reason: a common limiting distribution of sums of (independent) random variables have a limiting distribution that has power law tails. This was one of Mandelbrot's work horses in that power laws show up in nature, are self similar and represent a kind of "statistical" fractal [2]. There are other ways to generate power law tailed distributions as well, including having a random variable that has an exponentially increasing value with an exponentially decreasing probability of occurring, where each go at a different rate, say [3]. [1] https://en.wikipedia.org/wiki/Stable_distribution https://en.wikipedia.org/wiki/Stable_distribution [2] http://www.webofstories.com/play/benoit.mandelbrot/32;jsessionid=CCEF4615024059ED873729DFBCBBEFB7 http://www.webofstories.com/play/benoit.mandelbrot/32;jsessi... [3] https://www.math.uvic.ca/faculty/reed/PREpowerlaws.pdf https://www.math.uvic.ca/faculty/reed/PREpowerlaws.pdf

antognini 10y ago

My background is in astronomy, where power laws are ubiquitous. (There's a joke that when you get some data, you fit it to a power law. If that's not a good enough fit, you fit it to a broken power law.) The reason they're so common is that a power law is generally the distribution you get anytime you have a scale-invariant process.

klodolph 10y ago

A Galton box generates a binomial distribution. Most probability distributions are designed to model some kind of process. The time between clicks of a geiger counter is exponential, the number of clicks in a second is poisson. The amount of time your geiger counter lasts before it breaks and you have to buy a new one is weibull. That's the whole reason why we invented probability distributions in the first place: to model processes.

dekhn 10y ago
The Galton box generates a binomial distribution which when taken to the limit of n->infinity, produces a normal distribution.
- klodolph 10y ago
  This is incorrect, you need to take a limit as n->infinity while dividing by a factor of √n, otherwise the limit does not exist. But Galton boxes in practice are not very large, so they aren't very good approximations of normal distributions.

AlexCoventry 10y ago

What do you mean by all probability distributions? Any measurable function on a measurable space with Lebesgue integral 1 can be considered a probability distribution, so the answer to the technical interpretation of your question is no.

thomasahle 10y ago
Well, they do all have processes that approximately generate them, as by Monte Carlo sampling.

abetusk 10y ago

Power laws come up because they're the limiting distribution for a (common) class of processes. You can generate power law tailed distributions easily without dependence. The class of distributions that both power law tails and the Normal distribution fall under are called "Levy-stable distributions" [1]. Given the sum of independent, identically distributed (I.I.D.) random variables that converge to a stable distribution, the limiting distribution is Levy-stable (read, power law tails). If you the sum of the I.I.D. random variables has finite variance, the Gaussian distribution is the limiting distribution, which is a special case of the Levy-stable distributions. If each of the I.I.D. random variables has infinite second moment (e.g. infinite variance) and converges, then it converges to something with power law tails. We've been spoiled to think that the Gaussian's are the only or most common limiting distribution (thus the name "Normal") but Levy-stable distributions are another class of "common" distributions for the same reason: a common limiting distribution of sums of (independent) random variables have a limiting distribution that has power law tails. This was one of Mandelbrot's work horses in that power laws show up in nature, are self similar and represent a kind of "statistical" fractal [2]. There are other ways to generate power law tailed distributions as well, including having a random variable that has an exponentially increasing value with an exponentially decreasing probability of occurring, where each go at a different rate, say [3]. [1] https://en.wikipedia.org/wiki/Stable_distribution https://en.wikipedia.org/wiki/Stable_distribution [2] http://www.webofstories.com/play/benoit.mandelbrot/32;jsessionid=CCEF4615024059ED873729DFBCBBEFB7 http://www.webofstories.com/play/benoit.mandelbrot/32;jsessi... [3] https://www.math.uvic.ca/faculty/reed/PREpowerlaws.pdf https://www.math.uvic.ca/faculty/reed/PREpowerlaws.pdf