4 ms·
Of course. As others here point out, the hypervolume inside an n-dimensional hypersphere grows as the nth power of a linear increase in radius. In high dimensio
by cs702 9y ago
Of course. As others here point out, the hypervolume inside an n-dimensional hypersphere grows as the nth power of a linear increase in radius. In high dimensions, tiny increases in radius cause hypervolume to grow by more than 100%. The concentration of hypervolume is always highest at the edge.[0]
The theoretical tools (and intuitions) we have today for making sense of the distribution of data, developed over the past three centuries, break down in high dimensions. The fact that in high dimensions Gaussian distributions are not "clouds" but actually "soap bubbles" is a perfect example of this breakdown. Can you imagine trying to model a cloud of high-dimensional points lying on or near a lower-dimensional manifold with soap bubbles?
If the data is not only high-dimensional but also non-linearly entangled, we don't yet have "mental tools" for reasoning about it:
* https://medium.com/intuitionmachine/why-probability-theory-should-be-thrown-under-the-bus-36e5d69a34c9 https://medium.com/intuitionmachine/why-probability-theory-s...
* https://news.ycombinator.com/item?id=15620794 https://news.ycombinator.com/item?id=15620794
[0] See kgwgk's comment below.
- kgwgk 9y ago> Density is always highest at the edge. More precisely: it is the mass that is “concentrated” at the edge, not the density. In the Gaussian case the distribution “gets more and more dense in the middle” regardless of the number of dimensions. However, in high dimensions the volume in the middle is so low that essentially all the mass is close to the surface of the hypersphere.
- cs702 9y agoYes, you're right. For clarity's sake, I've updated my comment.
- ks1723 9y agoWhat i found quite surprising in that context, is that the volume of the n-dimensional ball for any finite fixed radius goes to zero as n goes to infinity (see, for example, section "high dimensions" of https://en.m.wikipedia.org/wiki/Volume_of_an_n-ball https://en.m.wikipedia.org/wiki/Volume_of_an_n-ball)
- cs702 9y agoThere are multiple good (intuitive) explanations for this here: https://math.stackexchange.com/questions/67039/why-does-volume-go-to-zero https://math.stackexchange.com/questions/67039/why-does-volu...
- rsp1984 9y agoThanks for this comment as it manages to explain something non-trivial in a very concise way, which is not something I find often.
- spookydooky 9y ago> the hypervolume inside an n-dimensional hypersphere grows as the nth power of a linear increase in radius. In high dimensions, tiny increases in radius cause hypervolume to grow by more than 100%. The concentration of hypervolume is always highest at the edge. > Of course. Elementary.