9 ms·
An n-ball Between n-balls
- robwwilliams 2y agoImpressive, helpful, and now time to rebuild my own embeddings so I can grasp that red n-ball with my new n-D hands.
- steventhedev 2y agoThis is a really good demonstration of the curse of dimensionality[0] [0]: https://en.m.wikipedia.org/wiki/Curse_of_dimensionality https://en.m.wikipedia.org/wiki/Curse_of_dimensionality
- ocean_moist 2y agoInteresting how this relates to LLMs scaling laws.
- deleted 2y ago[deleted]
- chadhutchins10 2y agoAnyone else click just to slide some animations?
- Hugsun 2y agoThis guy gets it!
- joaquincabezas 2y agowow discovering Hamming’s lecture was enough for me! so good
- blaufuchs 2y agoI know, I didn't realize he was alive in the 90s! Hearing him (sarcastically) say "now having 10 parameters isn't unusual, correct?" makes me wish he could've seen the 60B-parameter curve fitting we're doing nowadays.
- f1shy 2y agoI think he already knew something about it... it talks about AI. Maybe at the time 100k to 1M dimensions? A bright mind like his, could very good extrapolate to 2024.
- Scarblac 2y agoThe whole lecture series was about giving students a style of thinking that might hopefully prepare them for the future, without focusing on special knowledge (there were other courses for that). Two of the lectures were spent on building intuition for very high dimensionality (this one), and another on neural networks, because he thought there was a big chance they were going to be important. In the early 90s, not bad.
- f1shy 2y agoYes sir! I've just bookmarked the series... Priceless!
- Scarblac 2y agoThe whole series is on Youtube, awesome. The course was also the basis for his book _The Art of Doing Science and Engineering_ (1997). At first it takes some getting used to as you have the feeling it may be outdated, but it's about teaching a style of thinking. It's great.
- bt1a 2y agoI am struggling to juggle the balls in my mind. Are there any stepping-stone visual pieces like this to hopefully get me there? Very neat write-up, but I can't wait to share the realized absurdity of the red ball's green box eclipsing in our 3D intersection of the fully diagonalized 10D construct
- pfortuny 2y agoThe hypercube is the strange thing, not the red sphere. Placing the blue spheres tangent to the hypercube is an artificial construct which only “bounds” the red sphere in small dimensions. Our intuition is wrong because we think of the problem the wrong way (“the red sphere must be bounded by the box”, but there is no geometrical argument for that in n dimensions).
- badmintonbaseba 2y agoHow does the enclosed sphere's radius changes with the number of dimensions, if the enclosing spheres are the following: 2D: 3 mutually touching 2-spheres (circles) 3D: 4 mutually touching 3-spheres (or spheres) ... This variation of the problem doesn't rely on an artificial construct of a hypercube, I wonder if this yields a similarly unintuitive result.
- badmintonbaseba 2y agoIf my calculations are correct, then for this variation the enclosed n-sphere's radius converges to sqrt(2)-1 from below, and remains enclosed in the bounding hyper-tetrahedron.
- pfortuny 2y agoBuf, you may be right but I just cannot visualize it. It took me quite a while to do for the cube, imagine a tetrahedron. But you might be right.
- Hugsun 2y agoVery interesting, I've considered doing something similar with other regular polyhedra, like the n-simplex (the one you analyzed) and n-orthoplex. What was the side length in your calculation? did you find an equation for the size of the center n-ball?
- drdeca 2y agoWhy did I imagine that this would be about two shapes that are merely topologically n-balls, each having part of their boundary be incident with one of the two hemi(n-1)-spheres of the boundary of an n-ball (and otherwise not intersecting it)? (So like, in 3D, if you took some ball and two lumps of clay of different colors, and smooshed each piece of clay over half of the surface of the ball, with each of the two lumps of clay remaining topologically a 3-ball.) I don’t know that there would even be anything interesting to say about that.
- Hugsun 2y agoI can't tell you why you imagined that, but that's pretty funny nevertheless.
- mbowcut2 2y agoNumberphile did a video on this a while back. https://youtu.be/mceaM2_zQd8?si=0xcOAoF-Bn1Z8nrO https://youtu.be/mceaM2_zQd8?si=0xcOAoF-Bn1Z8nrO
- Imustaskforhelp 2y agoCan I just say how my mind is utterly blown by the animations
- Hugsun 2y agoThank you <3 The trigonometry involved was pretty intense at times.
- WhitneyLand 2y agoBoth ChatGpt 4.o and Claude failed to answer “…At what dimension would the red ball extend outside the box?” If anyone has o1-preview it’d be interesting to hear how well it does on this.
- V__ 2y agoThis was the prompt I gave o1-preview: > There is a geometric thought experiment that is often used to demonstrate the counterintuitive shape of high-dimensional phenomena. We start with a 4×4 square. There are four blue circles, with a radius of one, packed into the box. One in each corner. At the center of the box is a red circle. The red circle is as large as it can be, without overlapping the blue circles. When extending the construct to 3D, many things happen. All the circles are now spheres, the red sphere is larger while the blue spheres aren’t, and there are eight spheres while there were only four circles. > There are more than one way to extend the construct into higher dimensions, so to make it more rigorous, we will define it like so: An n-dimensional version of the construct consists of an n-cube with a side length of 4. On the midpoint between each vertex and the center of the n-cube, there is an n-ball with a radius of one. In the center of the n-cube there is the largest n-ball that does not intersect any other n-ball. > At what dimension would the red ball extend outside the box? Response: "[...] Conclusion: The red ball extends outside the cube when n≥10n≥10." It calculated it with a step-by-step explanation. This is the first time I'm actually pretty stunned. It analysed the problem, created an outline. Pretty crazy.
- Hugsun 2y agoI'd wager that it's in the training data.
- deleted 2y ago[deleted]
- eniwnenahg 2y agoMatlock, is that you?
- Hugsun 2y agoI am not Matlock, who is that?
- lisper 2y agoHe probably meant MacGyver. https://en.m.wikipedia.org/wiki/MacGyver https://en.m.wikipedia.org/wiki/MacGyver
- justtinker 2y agoNo, he really did mean Matlock. Grey hair guy in a white suit lecturing...Fictional Lawyer. https://www.alamy.com/stock-photo/matlock-tv-andy-griffith.html?sortBy=relevant https://www.alamy.com/stock-photo/matlock-tv-andy-griffith.h... Reboot on TV this year.
- lisper 2y agoI know who Matlock is. But then it makes no sense at all.
- Sharlin 2y agoA good way to conceptualize what’s going on is not the idea that balls become "spiky" in high dimensions – like the article says, balls are always perfectly symmetrical by definition. But it’s the box becoming spiky, "caltrop-shaped", its vertices reaching farther and farther out from the origin as the square root of dimension, while the centers of its sides remain at exactly +-1. And the 2^N surrounding balls are also getting farther from the origin, while their radius remains 1/2. Now it should be quite easy to imagine how the center ball gets more and more room until it grows out of the spiky box.
- pfortuny 2y agoExactly: a corner of a square covers 1/4 of that part of the plane. A corner of a cube covers 1/8, a corner of a hypercube in dimension n covers just 1/(2^n) of the space. But each side/face/hyperface divides the plane/space/n-dim space just in half.
- beretguy 2y agoOk, now i started to understand something. Thank you.
- jheriko 2y agoeven then the edges do not suddenly curve. its just all round a bad analogy.
- petters 2y agoVery good to see this as the top-voted comment. I completely agree that this seems like a more natural explanation of what is going on.
- ColinWright 2y agoBut there is another way to think of the high-dimensional balls where "spikey" is the right visualisation. Consider the volume of a cap. Take a plane that is 90% of the distance from the centre to the edge, and look at what percentage of the volume is "outside" that plane. When the dimension is high, that volume is negligible. And when the dimension is really high you can get quite close to the centre, and still the volume you cut off is very small. In our 3D world the closest thing that has this property is a spike. You can cut off quite close to the centre, and the volume excised is small. The sense in which a high-dimensional ball is not a spikey thing is in the symmetry, and the smoothness. So when you want to develop an intuition for a high dimensional ball you need to think of it as simultaneously symmetrical, smooth, and spikey. Then think of another five impossible things, and you can have breakfast.
- beretguy 2y agoI… can’t.
- ColinWright 2y agoFor other HN discussions of this phenomenon you can see some previous submissions of another article on it. That article doesn't have the nice animations, but it is from 14 years ago ... https://news.ycombinator.com/item?id=12998899 https://news.ycombinator.com/item?id=12998899 https://news.ycombinator.com/item?id=3995615 https://news.ycombinator.com/item?id=3995615 And from October 29, 2010: https://news.ycombinator.com/item?id=1846682 https://news.ycombinator.com/item?id=1846682
- mreid 2y agoI also wrote a short blog post about this topic back in 2009: https://mark.reid.name/blog/warning-high-dimensions.html https://mark.reid.name/blog/warning-high-dimensions.html It great to see these interesting math facts continue to be discussed and presented in new ways.
- Asraelite 2y agoI never understood the need to distinguish between "ball" and "sphere" in maths. Sure, one is solid and the other hollow, but why is that fact so important that you need to use a completely different word? As I understand it, you could replace every instance of "ball" in this article with "sphere" and it would still be correct. We don't have special words for the voluminous versions of other 3D shapes, so why do spheres need one?
- bmacho 2y ago> We don't have special words for the voluminous versions of other 3D shapes, so why do spheres need one? Topologists don't need them because they already have ball and sphere. In analysis, I can imagine them calling full hyperrectangles "brick", and empty hyperrectangles "box", but both words start with "b", so there is no shorthand for them on paper. I^n and ∂I^n are just fine.
- NeoTar 2y agoAnd, indeed, it’s probably in topology where the distinction between sphere and ball is most important.
- Hugsun 2y ago> you could replace every instance of "ball" in this article with "sphere" and it would still be correct. I completely agree. N-ball is maybe more mathematically precise. It might rule out some strange edge case I couldn't think of. I chose it mostly for stylistic reasons. n-ball is shorter than n-sphere, and ball is a more playful term. I do however understand the need to define the two object classes, and see little issue with giving them distinct names.
- deleted 2y ago[deleted]
- NeoTar 2y agoWe do for 2D though - Circle and disc. And in 4D plus we have the 3-sphere and 4-ball, etc. And for 3D shapes have Torus, and, well, ‘solid-torus’.