4 ms·

For high-dimensional spheres, most of the volume is in the "shell", ie near the boundary [0]. This sort of makes sense to me, but I don't know how to square tha

by FabHK 2y ago

For high-dimensional spheres, most of the volume is in the "shell", ie near the boundary [0]. This sort of makes sense to me, but I don't know how to square that with the observation in the article that most of the surface area is near the equator. (In particular, by symmetry, it's near any equator; so, one would think, in their intersection. That is near the centre, though, not the shell.) Anyway. Never buy a high-dimensional orange, it's mostly rind. [0] https://www.math.wustl.edu/~feres/highdim https://www.math.wustl.edu/~feres/highdim

hansvm 2y ago
It's basically the same idea in both cases. Power laws warp anything "slightly bigger" into dominating everything else when the power is big enough. There's a bit more stuff near the outside than the inside, so with a high enough dimension the volume is in the rind. Similarly, the equator is a bit bigger than the other slices, so with enough dimensions its surface area dominates.
- WiSaGaN 2y ago
  Yes, this seems to be the result of the standard Euclidean metric rather than the high dimension itself. I guess most people assuming the metric to be Euclidean, so it's ok.
  - hansvm 2y ago
    The conditions you need for that to be true are substantially weaker than being Euclidean (though, when people are talking about "weird" behavior in high-dimensional spaces nowadays, it's in the context of ML and Euclidean stuff anyway). If you have a meaningful notion of dimension (basic properties like <1,0> being different from <0,1> and <1,0> being closer to <1,1> than <0,1>) and and don't have discretized shenanigans (which would collapse the inequality I'm exploiting to a strict equality, with some sort of 1^n=1 behavior) then the natural measure induced by the metric in question will exhibit the described behavior. You can easily verify that for every metric represented in scipy or whatnot, and a proper proof isn't too much more work.
youoy 2y ago
If you like ML, this is also related with the results of this paper [0], where they show that learning in high dimensions amounts to extrapolation, as opposed to interpolation. Intuitively I think of this as the fact that points in the sphere are convexly independent, and most of the volume of the ball is near the boundary. [0] https://arxiv.org/abs/2110.09485 https://arxiv.org/abs/2110.09485
deleted 2y ago
[deleted]

by FabHK 2y ago

hansvm 2y ago

It's basically the same idea in both cases. Power laws warp anything "slightly bigger" into dominating everything else when the power is big enough. There's a bit more stuff near the outside than the inside, so with a high enough dimension the volume is in the rind. Similarly, the equator is a bit bigger than the other slices, so with enough dimensions its surface area dominates.

WiSaGaN 2y ago
Yes, this seems to be the result of the standard Euclidean metric rather than the high dimension itself. I guess most people assuming the metric to be Euclidean, so it's ok.
- hansvm 2y ago
  The conditions you need for that to be true are substantially weaker than being Euclidean (though, when people are talking about "weird" behavior in high-dimensional spaces nowadays, it's in the context of ML and Euclidean stuff anyway). If you have a meaningful notion of dimension (basic properties like <1,0> being different from <0,1> and <1,0> being closer to <1,1> than <0,1>) and and don't have discretized shenanigans (which would collapse the inequality I'm exploiting to a strict equality, with some sort of 1^n=1 behavior) then the natural measure induced by the metric in question will exhibit the described behavior. You can easily verify that for every metric represented in scipy or whatnot, and a proper proof isn't too much more work.

youoy 2y ago

If you like ML, this is also related with the results of this paper [0], where they show that learning in high dimensions amounts to extrapolation, as opposed to interpolation. Intuitively I think of this as the fact that points in the sphere are convexly independent, and most of the volume of the ball is near the boundary. [0] https://arxiv.org/abs/2110.09485 https://arxiv.org/abs/2110.09485

deleted 2y ago

[deleted]