5 ms·

Reading the article I couldn't stop thinking: Isn't 3D space perhaps being mapped by a "crumpled" 2D space filling surface? Real world environments are rarely

Reading the article I couldn't stop thinking: Isn't 3D space perhaps being mapped by a "crumpled" 2D space filling surface? Real world environments are rarely really 3D, and a 2D surface is perhaps the sweet spot in complexity/degrees of freedom for most practical cases? This would explain why 3D mazes are so much more disorienting than 2D mazes, for example.

JabavuAdams 5y ago
Sorry if I'm Mx-splaining -- I don't know your background, but I'm interested in/working on this stuff. Your crumpled space-filling surface makes me think of manifolds. Machine-learning researchers often speak loosely of e.g. the "manifold"* of possible images of naturalistic scenes embedded within the space of all possible images. The networks are presumably learning a lower-dimensional representation of the world than the dimension of all possible combinations of sense impressions. If you have a 100 pixel by 100 pixel image and each pixel can have 256 intensity levels, then that's 256 to the power of ten thousand possible distinct images. If each image is a point in an abstract space of all possible images, then that space has 256 to the ten thousand dimensions. But the vast, vast, vast majority of the volume of that space corresponds to images that just look like static to humans. So the thinking is that we internally represent images as some learned non-linear transformation to a much lower dimensional set of features that actually correspond to stuff we experience / see. A really simple canonical example is the Swiss Roll dataset. You only need two parameters (numbers) to specify it fully, but you can embed it (nonlinearly) in a 3D space. http://people.cs.uchicago.edu/~dinoj/manifold/swissroll.html http://people.cs.uchicago.edu/~dinoj/manifold/swissroll.html In terms of neuroscience, there are competing ideas (as always). There's a body of work that tries to show that as we learn, the brain encodes our high-dimensional sense-impressions in the lowest possible (most efficient) internal representations. However, some recent work seems to imply that instead the brain uses as high a dimension as possible, but up to some limit that demarcates the transition between being differentiable and not differentiable. They found a beautiful power-law: https://www.biorxiv.org/content/10.1101/374090v1.full https://www.biorxiv.org/content/10.1101/374090v1.full * When ML researchers speak of such a manifold, it's kind of loosey-goosey because a set that includes isolated points that don't have a continuous region around them aren't actually manifolds. In contrast, in the computer graphics and meshing literature people speak of non-manifold geometry which is isolated points and lines that you can't triangulate with 2d or 3d elements. I.e. 1d or 0d elements.
- whatshisface 5y ago
  >When ML researchers speak of such a manifold, it's kind of loosey-goosey because a set that includes isolated points that don't have a continuous region around them aren't actually manifolds. So they mean "subset" when they say "manifold?"
  - JabavuAdams 5y ago
    I suppose so, but they really are trying to convey that all the points lie close to some lower-dimensional crumpled surface. So a subset that was e.g. just a lattice in the high-dimensional space wouldn't fit this mental image. E.g. Generate 3d points that are on a 2d plane +- some small random offset normal to the plane. If the points are isolated, without each having a local neighbourhood, it's not technically a manifold. But, you could describe the data-set as lying on or near a plane to within some tolerance. So they're trying to describe something that is much more specific / strongly constrained than an arbitrary subset, but it doesn't meet the very stringent (and frankly idealized) requirements of a manifold. I wonder whether there is a math-object to describe this? EDIT> Maybe it's just a matter of saying "close to some lower-dimensional manifold", rather than "on a lower-dimensional manifold."
    - whatshisface 5y ago
      >I wonder whether there is a math-object to describe this? Here is how I would phrase it: there is a probability distribution in the higher-dimensional space that expresses P(this image | given that it's a natural image). The level sets of the probability distribution are manifolds.
- TheCoreh 5y ago
  > Sorry if I'm Mx-splaining -- I don't know your background Not at all! All of this is far more advanced than my knowledge on the topic, and very interesting! Thanks for sharing That idea of representing in the highest dimensionality possible, with some constraint is also very interesting. In that case perhaps the 3D space is being represented in a higher dimensional form that makes it more convenient for some neural processing purpose (e.g. just like we use homogenous coordinates) The 2D-2D case is then just a happy coincidence where the highest representation that makes sense maps 1 to 1 with the actual data.

JabavuAdams 5y ago

Sorry if I'm Mx-splaining -- I don't know your background, but I'm interested in/working on this stuff. Your crumpled space-filling surface makes me think of manifolds. Machine-learning researchers often speak loosely of e.g. the "manifold"* of possible images of naturalistic scenes embedded within the space of all possible images. The networks are presumably learning a lower-dimensional representation of the world than the dimension of all possible combinations of sense impressions. If you have a 100 pixel by 100 pixel image and each pixel can have 256 intensity levels, then that's 256 to the power of ten thousand possible distinct images. If each image is a point in an abstract space of all possible images, then that space has 256 to the ten thousand dimensions. But the vast, vast, vast majority of the volume of that space corresponds to images that just look like static to humans. So the thinking is that we internally represent images as some learned non-linear transformation to a much lower dimensional set of features that actually correspond to stuff we experience / see. A really simple canonical example is the Swiss Roll dataset. You only need two parameters (numbers) to specify it fully, but you can embed it (nonlinearly) in a 3D space. http://people.cs.uchicago.edu/~dinoj/manifold/swissroll.html http://people.cs.uchicago.edu/~dinoj/manifold/swissroll.html In terms of neuroscience, there are competing ideas (as always). There's a body of work that tries to show that as we learn, the brain encodes our high-dimensional sense-impressions in the lowest possible (most efficient) internal representations. However, some recent work seems to imply that instead the brain uses as high a dimension as possible, but up to some limit that demarcates the transition between being differentiable and not differentiable. They found a beautiful power-law: https://www.biorxiv.org/content/10.1101/374090v1.full https://www.biorxiv.org/content/10.1101/374090v1.full * When ML researchers speak of such a manifold, it's kind of loosey-goosey because a set that includes isolated points that don't have a continuous region around them aren't actually manifolds. In contrast, in the computer graphics and meshing literature people speak of non-manifold geometry which is isolated points and lines that you can't triangulate with 2d or 3d elements. I.e. 1d or 0d elements.

whatshisface 5y ago
>When ML researchers speak of such a manifold, it's kind of loosey-goosey because a set that includes isolated points that don't have a continuous region around them aren't actually manifolds. So they mean "subset" when they say "manifold?"
- JabavuAdams 5y ago
  I suppose so, but they really are trying to convey that all the points lie close to some lower-dimensional crumpled surface. So a subset that was e.g. just a lattice in the high-dimensional space wouldn't fit this mental image. E.g. Generate 3d points that are on a 2d plane +- some small random offset normal to the plane. If the points are isolated, without each having a local neighbourhood, it's not technically a manifold. But, you could describe the data-set as lying on or near a plane to within some tolerance. So they're trying to describe something that is much more specific / strongly constrained than an arbitrary subset, but it doesn't meet the very stringent (and frankly idealized) requirements of a manifold. I wonder whether there is a math-object to describe this? EDIT> Maybe it's just a matter of saying "close to some lower-dimensional manifold", rather than "on a lower-dimensional manifold."
  - whatshisface 5y ago
    >I wonder whether there is a math-object to describe this? Here is how I would phrase it: there is a probability distribution in the higher-dimensional space that expresses P(this image | given that it's a natural image). The level sets of the probability distribution are manifolds.
TheCoreh 5y ago
> Sorry if I'm Mx-splaining -- I don't know your background Not at all! All of this is far more advanced than my knowledge on the topic, and very interesting! Thanks for sharing That idea of representing in the highest dimensionality possible, with some constraint is also very interesting. In that case perhaps the 3D space is being represented in a higher dimensional form that makes it more convenient for some neural processing purpose (e.g. just like we use homogenous coordinates) The 2D-2D case is then just a happy coincidence where the highest representation that makes sense maps 1 to 1 with the actual data.