6 ms·
One big problem with the conclusion is that intuitions from low dimensional spaces often don’t carry over to high dimensional spaces. e.g. common example is how
by ajtulloch 8y ago
One big problem with the conclusion is that intuitions from low dimensional spaces often don’t carry over to high dimensional spaces. e.g. common example is how the volume of the unit hypersphere intersected with hypercube ratio goes to zero. One funny thing I saw once was something like “the real curse of dimensionality is how different optimization in high dimensional spaces is compared to low dimensional spaces”.
- Terr_ 8y ago> One big problem with the conclusion is that intuitions from low dimensional spaces often don’t carry over to high dimensional spaces. e.g. common example is how the volume of the unit hypersphere intersected with hypercube ratio goes to zero. Is that un-intuitive, or am I looking at it the "wrong" way? I'm imagining the overlap of a 2D square/unit-circle versus the overlap of a 3D cube/unit-sphere, and from those two data points there's already a downward trend. I mean, the 2D case is really just the single cross-sectional slice from the 3D case through the middle, the one that that has the greatest possible overlap. All other possible slices will less overlap. Following that logic, a 3D square/cube overlap is likewise the the "middle cross-section of greatest overlap" for a 4D scenario, etc.
- ubershmekel 8y agoDownward trend doesn't imply trending to zero. Would your intuition suggest a trend to zero? Even if it does - you can build intuitions to explain unintuitive results. Unituitive is subjective and a similar word would be "surprising". Because you can look at data to explain a surprising result - it doesn't make the initial result less surprising.
- Strilanc 8y agoTry this one. I give you a megabox: a million dimensional hypercube spanning one meter along each axis. I also provide a bunch of megaspheres, million-dimensional hyperspheres scaled to be 0.99 meters in diameter. How many megaspheres can you fit into the megabox?
- sudhirj 8y agoThe first answer in my head is 1, but I'm guessing this is more akin to 'how many circles can you fit in a sphere?'?
- Strilanc 8y agoThe spheres and the cubes are of the same dimension, so it's not like trying to put flat things inside of a thing with volume. (Technically I should have said million dimensional balls instead of spheres. Sorry if that was confusing.)
- sudhirj 8y agoThen isn’t the answer 1? For 2 dimensions, one circle fits in a square, in three dimensions one sphere fits in a cube. Should be the same for any value of n?
- Strilanc 8y agoConsider that the distance between opposite corners of a square is √2 meters, but for a cube it's √3 meters. In general, for an n-dimensional hypercube of diameter 1m, the exactly-opposite corners are √n meters apart. Meaning, in a million-dimensional meter-diameter hypercube, exactly-opposite corners will be a kilometer apart. There's a lot more diagonal wiggle room than you think.
- therein 8y agoThat blew my mind but it still sounds like it would generalize, wouldn't it?
- jstanley 8y agoI'd love to hear the correct answer to this, with explanation.
- 8y ago
- dTal 8y agoDon't forget the 1D case, which has a 100% overlap.
- pbhjpbhj 8y agoDoesn't a sphere require all points to be separated from the centre point by the same distance r, where r>0? A point isn't a sphere.
- detaro 8y agoReally depends on who you are asking for a definition. E.g. there is the term "point sphere" for the special case of a sphere that is a point.
- pbhjpbhj 8y agoIs there any interesting result that comes from casting a point as being any shape (presumably the point-sphere is also a point-dodecahedron, etc.)?
- chestervonwinch 8y agoDefine a topology on any set you'd like and a sphere is just the set of limit points of any open ball.
- gus_massa 8y agoAre you taking about the orange peel, the orange juicy part or the whole fruit? If you consider only the "orange peel", if the center is 0 and r=1, then the 1D version is just two points {-1} and {1}, not only one point. If you consider only the "whole orange" then the 1D case is the segment [-1, 1], if the center is 0 and r=1 In both case it is equal to the 1D "square" equivalent.
- dTal 8y agoThe 1D case is a line segment, not a point. A point is the 0D case and indeed a degenerate case.
- Zophike1 8y ago> One big problem with the conclusion is that intuitions from low dimensional spaces often don’t carry over to high dimensional spaces. e.g. common example is how the volume of the unit hypersphere intersected with hypercube ratio goes to zero. One funny thing I saw once was something like “the real curse of dimensionality is how different optimization in high dimensional spaces is compared to low dimensional spaces”. How so usually in Pure Mathematics(Analytic area's) everything done in R^{1} R^{2} is usually generalized to R^{N} and much of the intuition carriers over. So how does it fail here in the context of Machine Learning ? Note: I know nothing about ML :(
- edraferi 8y agoAll the mechanics scale like you’d expect. You have bigger matrices containing more observations with more parameters per observation. Keep the dimensions aligned properly and everything works out just like it would with smaller vectors. The discrepancies appear when you start dealing with probabilistic stuff. Suddenly everything’s conditioned by everything else, and your complexity skyrockets. Example: it’s easy to find people with about average height and weight, but the more physical characteristics you measure (increasing dimensionality) the more like you are to find something statistically unusual, eg very long fingers. Very interesting Air Force research on this [0] WRT to optimization, higher dimensions can help avoid the local minima problem. When you have 1000s if possible directions to go, it’s unlikely that none of them will yield a better answer. [0] https://www.thestar.com/news/insight/2016/01/16/when-us-air-force-discovered-the-flaw-of-averages.html https://www.thestar.com/news/insight/2016/01/16/when-us-air-...