7 ms·
Benford's law, Zipf's law and other power laws tend to fascinate but they are a manifestation of something really simple. It's how things behave when they are e
by BenoitEssiambre 7y ago
Benford's law, Zipf's law and other power laws tend to fascinate but they are a manifestation of something really simple. It's how things behave when they are evenly distributed over multiplicative operations instead of additive operations. By this I mean things are as likely to be x% bigger than x% smaller instead of the more common definition of evenly distributed (things x unit larger are as likely as x unit smaller).
Being evenly distributed over multiplication is natural for many measures. I think it is considered the most even, "maximum entropy" distribution for things that can't be negative.
The most even, maximum entropy distribution for an unbounded (-infinity to +infinity) prior is an unbounded uniform distribution but you can't apply it, for example, in the case of the size of objects since it would imply that negative sizes are as likely as positive sizes (and objects don't have negative sizes).
In general position parameters tend to be uniformly distributed over addition/subtraction whereas scale parameters tend to be evenly distributed over multiplication (uniform over log(x)). It's the reason some graphs use log axis. For Bayesians, it's a fairly straightforwards concept. See also improper priors (https://en.wikipedia.org/wiki/Prior_probability#Improper_priors https://en.wikipedia.org/wiki/Prior_probability#Improper_pri...)
- vackosar 7y agoListen to the paper on this topic here you can: https://www.youtube.com/watch?v=pGBcBeX1PSA https://www.youtube.com/watch?v=pGBcBeX1PSA :)
- lonelappde 7y agoAs the article explains, benford's law applies to distributions over values that evolve over time with multiplicative fluctuations, not multiplicative distributions themselves. A simple uniform distribution over [0,n] obeys benford's law, averages over any range of N.
- BenoitEssiambre 7y agoTime doesn't have to be a component. Also data distributed uniformly in a range would not obey Benford's law. Each digit would appear with equal frequency.
- susam 7y ago> A simple uniform distribution over [0,n] obeys benford's law A simple uniform distribution does not obey Benford's law because each digit from 1 to 9 is equally likely to be the leading significant digit. Thus each digit occurs as the leading digit with 11.1% probability. See https://news.ycombinator.com/item?id=22340018 https://news.ycombinator.com/item?id=22340018 for an example.
- dmurray 7y agoWhat if N is 19?
- susam 7y agoThe larger the value of N is, the more the experiments agree with the Benford's law.
- foo101 7y agoIt does not matter what N you choose. The experimental results are always an approximation of Benford's law. The approximation gets better and better as N becomes larger and larger. The fact that you do not see Benford's law working with N = 19 is not in anyway a flaw in the way the experimental results are set up. It just means N is too small to approximate Benford's law closely.
- EGreg 7y agoOf course it does, because there is going to be a maximum of the range A, B. Even if the maximum itself would have a uniform distribution, the fact is that all numbers drawn from the samples will be below it. So the probability that the number starts with a 5 (in decimal representation) is sort of like the probability of the sum over all N of: (Maximum > N) (Number starts with 5 given that Maximum is N) The thing is that when N is anything less than 9,999 etc. the second probability IS NOT 1/10. It is skewed towards the lower numbers due to the most significant digit being less than 9 more frequently!!! So this is the real explanation. And I just came up with it on the spot while writing this. Yay! Hehe
- 7y ago
- hyperbovine 7y agoThe max entropy distribution over (-oo,+oo) is usually taken to be either Gaussian or Laplace (two sided exponential), depending on moment conditions. There is no such thing as a uniform distribution over the real line.
- baxtr 7y agoSide note: great how you spell infinity. Never saw this before. So simple and smart
- polyphonicist 7y agoCame here to say the same thing. Certainly haven't seen such depiction of infinity earlier. But this is a nice way. I am going to use it myself.
- mkl 7y agoIt's used as the symbol for infinity in Sympy, and probably other (and older) computer algebra systems.
- BenoitEssiambre 7y agoOf course there is. It's an improper prior (doesn't sum up to one), see link above. Theoretically the Gaussian is only maximum entropy given a mean and stddev. An infinite uniform is maxent without any knowledge of mean and stddev. In practice though you can use a wide Gaussian if it is more mathematically convenient and get about the same results.
- deleted 7y ago[deleted]
- hyperbovine 7y agoThe entropy of any improper prior diverges. There is no point in arguing over which one maximizes entropy; they all do, vacuously. Max entropy is not useful for discriminating amongst probability distributions unless you require them to actually be probability distributions.