4 ms·
Your shock is warranted. It's actually a very deep question, called the Zipf mystery: https://www.youtube.com/watch?v=fCn8zs912OE https://www.youtube.com/watch?
by sillysaurusx 7y ago
Your shock is warranted. It's actually a very deep question, called the Zipf mystery: https://www.youtube.com/watch?v=fCn8zs912OE https://www.youtube.com/watch?v=fCn8zs912OE
No one really knows why it's true. It's also true across physical constants, not just artificial quantities made up by humans.
The same type of distribution pops up in language, in nature, in numbers, popularity of opening chess moves, everywhere.
- saagarjha 7y agoZipf’s Law is related, but not quite the same thing: https://en.wikipedia.org/wiki/Zipf's_law https://en.wikipedia.org/wiki/Zipf's_law
- ColinWright 7y ago> No one really knows why it's true. Any distribution of numbers that is "scale free" follows Benford's Law. This is extremely well understood. The distribution of the areas of lakes, heights of trees, the proportion of the answers that start with the digit "1" should be the same no matter what units you use. You can measure heights in barleycorns, metres, inches, or cubits, the proportion of answers that start with the digit "1" should remain constant across those different units. Grind through the sums and Benford's Law pops out. And Benford's Law is not Zipf's Law. Zipf's Law is also well-understood when you look at coding theory, and how we would expect language (and mutable codes in general) to change so that more frequently used "atoms" are "shorter. So I don't really know what you're claiming here, but on the surface of it, with things I've studied, your comment seems mostly wrong.
- emmelaich 7y agoAt first I thought Zipf's had nothing to do with Benford's. But then .. https://terrytao.wordpress.com/2009/07/03/benfords-law-zipfs-law-and-the-pareto-distribution/ https://terrytao.wordpress.com/2009/07/03/benfords-law-zipfs...
- xelxebar 7y ago> No one really knows why it's true. This is what is so bizzare. We do know why it's true; the mystery is just pumped up in the press. I have only skimmed the paper, but apparently Belevitch [0] showed (in 1959!) that Zipf's law and Mandelbrot's Law [1] are essentially just first-order and second-order approximations, respectively, for a large class of probability distributions, so no wonder it is so ubiquitous. [0] Belevitch, V. (1959). On the statistical laws of linguistic distributions. Annales de la Societe Scientifique de Bruxelles, 73(3), 301–326. [1] https://en.wikipedia.org/wiki/Zipf%E2%80%93Mandelbrot_law https://en.wikipedia.org/wiki/Zipf%E2%80%93Mandelbrot_law