6 ms·

How Animals Map 3D Spaces Surprises Brain Researchers

90 points by jonbaer 5y ago

pharke 5y ago
So are the hexagonal latices for 2D grid cells just an artifact of whatever sphere packing[0] algorithm the neurons are using? [0]https://en.wikipedia.org/wiki/Sphere_packing https://en.wikipedia.org/wiki/Sphere_packing
- JabavuAdams 5y ago
  I love sphere packing, but we don't know that neurons do sphere packing. Check out Shimada et. al. Bubble Mesh for some fun sphere-packing algos.
  - pharke 5y ago
    I find this paragraph from the article intriguing > But the grid cells’ firing wasn’t entirely random either. Instead, there was local order: For each grid cell, the places where it fired weren’t arranged in a perfect periodic lattice, but the distances between them were too regular to be merely a matter of chance. Rather than the neat stack of oranges, the researchers were seeing something similar but less orderly, more like marbles filling a box. “They’re always stuck in some local minimum, such that there is not a lattice,” Ulanovsky said. “On the other hand, the local distances there are fixed, because all the [marbles] are sort of touching their neighbors.”
TheCoreh 5y ago
Reading the article I couldn't stop thinking: Isn't 3D space perhaps being mapped by a "crumpled" 2D space filling surface? Real world environments are rarely really 3D, and a 2D surface is perhaps the sweet spot in complexity/degrees of freedom for most practical cases? This would explain why 3D mazes are so much more disorienting than 2D mazes, for example.
- JabavuAdams 5y ago
  Sorry if I'm Mx-splaining -- I don't know your background, but I'm interested in/working on this stuff. Your crumpled space-filling surface makes me think of manifolds. Machine-learning researchers often speak loosely of e.g. the "manifold"* of possible images of naturalistic scenes embedded within the space of all possible images. The networks are presumably learning a lower-dimensional representation of the world than the dimension of all possible combinations of sense impressions. If you have a 100 pixel by 100 pixel image and each pixel can have 256 intensity levels, then that's 256 to the power of ten thousand possible distinct images. If each image is a point in an abstract space of all possible images, then that space has 256 to the ten thousand dimensions. But the vast, vast, vast majority of the volume of that space corresponds to images that just look like static to humans. So the thinking is that we internally represent images as some learned non-linear transformation to a much lower dimensional set of features that actually correspond to stuff we experience / see. A really simple canonical example is the Swiss Roll dataset. You only need two parameters (numbers) to specify it fully, but you can embed it (nonlinearly) in a 3D space. http://people.cs.uchicago.edu/~dinoj/manifold/swissroll.html http://people.cs.uchicago.edu/~dinoj/manifold/swissroll.html In terms of neuroscience, there are competing ideas (as always). There's a body of work that tries to show that as we learn, the brain encodes our high-dimensional sense-impressions in the lowest possible (most efficient) internal representations. However, some recent work seems to imply that instead the brain uses as high a dimension as possible, but up to some limit that demarcates the transition between being differentiable and not differentiable. They found a beautiful power-law: https://www.biorxiv.org/content/10.1101/374090v1.full https://www.biorxiv.org/content/10.1101/374090v1.full * When ML researchers speak of such a manifold, it's kind of loosey-goosey because a set that includes isolated points that don't have a continuous region around them aren't actually manifolds. In contrast, in the computer graphics and meshing literature people speak of non-manifold geometry which is isolated points and lines that you can't triangulate with 2d or 3d elements. I.e. 1d or 0d elements.
  - whatshisface 5y ago
    >When ML researchers speak of such a manifold, it's kind of loosey-goosey because a set that includes isolated points that don't have a continuous region around them aren't actually manifolds. So they mean "subset" when they say "manifold?"
MR4D 5y ago
How birds can fly into a tree at speed, and land on a branch without poking an eye out (or anything else) several times per day is amazing to me. "Bird-brain" may be an insult to people, but I'd bet their 3-D processing capabilities blow away an RTX 3090 - especially when considering power use and heat. We have a lot left to learn from them.
- trhway 5y ago
  the trick i observed crows doing - somewhat similar to the opening of the Cobra maneuvers - they come about a foot below branch or whatever their landing target is and by quickly angling up they extinguish their horizontal speed by trading it for that foot of the height - ie. they get 0 horizontal speed right over the branch. Thus they avoid the overwise would be needed horizontal speed slowdown in the level flight and associated stall risk.
- deleted 5y ago
  [deleted]
- clairity 5y ago
  also amazing are gibbons. they literally fly through 3d tree-space using branches that flex generously, making the split-second routing calculations they’re making seem unbelievable.
  - jrootabega 5y ago
    Gibbons are actually not that good at tree swinging, they have just developed paracausal abilities. When they miss a branch, they actually fall out of the spacetime continuum entirely, and to us it seems it never happened.
    - clairity 5y ago
      > “Gibbons are actually not that good at tree swinging…” having done research in this area, i’m fairly confident in my assertion, but am open to any new information/evidence you may provide. the rest of your statement provides one evolutionary mechanism by which gibbons became good brachiators, not evidence that they’re currently not.
doodlebugging 5y ago
After reading this article I have decided that the researchers earlier conclusions about how the rats keep track of their environment is suffering from a serious defect introduced by the researchers at the start. Their earlier work involved studying rat and bat navigation in unnatural environments - a "2D" space which was really a 3D space constrained to be along a single level. This forced or even allowed the rat brains to dumb things down to a situation where they had no need or opportunity to consider things outside that unnaturally limited space and so their brains naturally optimized for that simplistic situation. You have dumbed down their environment to the point where you have established a minimal ability that they need to master in order to function and navigate that environment. Then when you discover that their memory encodes navigational information from this dumbed-down, unnatural environment in a regularized grid, this should be no surprise. They are minimizing the energy required to thrive in that environment. Once you allow them to navigate in a more natural environment you should expect to see this regular grid disappear since their options for reaching a destination are no longer constrained to a single path along most of the route to their destination. Their brains will have to incorporate many more clues to guide their route selection and those clues could be time-varying as in the case where they are tracking the source of an odor as they search for food so they will need to monitor air flow and direction, odor intensity, potential obstacles between themselves and the source, alternate routes which present themselves as they move through space, threat detection from predators or bait traps, etc. In short, there so many variables that will need to be evaluated once you remove the flat plane constraints that led to their grid discovery that it should be no surprise to discover that a more complex environment uses different optimizations, some of which are encoded in a regularized notation and others, probably situational events on the path-picking decision tree, are more random.
- bckr 5y ago
  I think it's important to see all of these situations. The very fact that the brain optimizes its maps based on constrained environments is awesome. Also, the technology had to be built to do the previous 2d studies, which then bootstrapped the 3d studies. Knowing that under certain circumstances the neural maps had a very "clean architecture" may have saved a lot of time and effort once these more random 3d structures were being analyzed.

90 points by jonbaer 5y ago

So are the hexagonal latices for 2D grid cells just an artifact of whatever sphere packing[0] algorithm the neurons are using? [0]https://en.wikipedia.org/wiki/Sphere_packing https://en.wikipedia.org/wiki/Sphere_packing

JabavuAdams 5y ago
I love sphere packing, but we don't know that neurons do sphere packing. Check out Shimada et. al. Bubble Mesh for some fun sphere-packing algos.
- pharke 5y ago
  I find this paragraph from the article intriguing > But the grid cells’ firing wasn’t entirely random either. Instead, there was local order: For each grid cell, the places where it fired weren’t arranged in a perfect periodic lattice, but the distances between them were too regular to be merely a matter of chance. Rather than the neat stack of oranges, the researchers were seeing something similar but less orderly, more like marbles filling a box. “They’re always stuck in some local minimum, such that there is not a lattice,” Ulanovsky said. “On the other hand, the local distances there are fixed, because all the [marbles] are sort of touching their neighbors.”

TheCoreh 5y ago

Reading the article I couldn't stop thinking: Isn't 3D space perhaps being mapped by a "crumpled" 2D space filling surface? Real world environments are rarely really 3D, and a 2D surface is perhaps the sweet spot in complexity/degrees of freedom for most practical cases? This would explain why 3D mazes are so much more disorienting than 2D mazes, for example.

JabavuAdams 5y ago
Sorry if I'm Mx-splaining -- I don't know your background, but I'm interested in/working on this stuff. Your crumpled space-filling surface makes me think of manifolds. Machine-learning researchers often speak loosely of e.g. the "manifold"* of possible images of naturalistic scenes embedded within the space of all possible images. The networks are presumably learning a lower-dimensional representation of the world than the dimension of all possible combinations of sense impressions. If you have a 100 pixel by 100 pixel image and each pixel can have 256 intensity levels, then that's 256 to the power of ten thousand possible distinct images. If each image is a point in an abstract space of all possible images, then that space has 256 to the ten thousand dimensions. But the vast, vast, vast majority of the volume of that space corresponds to images that just look like static to humans. So the thinking is that we internally represent images as some learned non-linear transformation to a much lower dimensional set of features that actually correspond to stuff we experience / see. A really simple canonical example is the Swiss Roll dataset. You only need two parameters (numbers) to specify it fully, but you can embed it (nonlinearly) in a 3D space. http://people.cs.uchicago.edu/~dinoj/manifold/swissroll.html http://people.cs.uchicago.edu/~dinoj/manifold/swissroll.html In terms of neuroscience, there are competing ideas (as always). There's a body of work that tries to show that as we learn, the brain encodes our high-dimensional sense-impressions in the lowest possible (most efficient) internal representations. However, some recent work seems to imply that instead the brain uses as high a dimension as possible, but up to some limit that demarcates the transition between being differentiable and not differentiable. They found a beautiful power-law: https://www.biorxiv.org/content/10.1101/374090v1.full https://www.biorxiv.org/content/10.1101/374090v1.full * When ML researchers speak of such a manifold, it's kind of loosey-goosey because a set that includes isolated points that don't have a continuous region around them aren't actually manifolds. In contrast, in the computer graphics and meshing literature people speak of non-manifold geometry which is isolated points and lines that you can't triangulate with 2d or 3d elements. I.e. 1d or 0d elements.
- whatshisface 5y ago
  >When ML researchers speak of such a manifold, it's kind of loosey-goosey because a set that includes isolated points that don't have a continuous region around them aren't actually manifolds. So they mean "subset" when they say "manifold?"

MR4D 5y ago

How birds can fly into a tree at speed, and land on a branch without poking an eye out (or anything else) several times per day is amazing to me. "Bird-brain" may be an insult to people, but I'd bet their 3-D processing capabilities blow away an RTX 3090 - especially when considering power use and heat. We have a lot left to learn from them.

trhway 5y ago
the trick i observed crows doing - somewhat similar to the opening of the Cobra maneuvers - they come about a foot below branch or whatever their landing target is and by quickly angling up they extinguish their horizontal speed by trading it for that foot of the height - ie. they get 0 horizontal speed right over the branch. Thus they avoid the overwise would be needed horizontal speed slowdown in the level flight and associated stall risk.
deleted 5y ago
[deleted]
clairity 5y ago
also amazing are gibbons. they literally fly through 3d tree-space using branches that flex generously, making the split-second routing calculations they’re making seem unbelievable.
- jrootabega 5y ago
  Gibbons are actually not that good at tree swinging, they have just developed paracausal abilities. When they miss a branch, they actually fall out of the spacetime continuum entirely, and to us it seems it never happened.
  - clairity 5y ago
    > “Gibbons are actually not that good at tree swinging…” having done research in this area, i’m fairly confident in my assertion, but am open to any new information/evidence you may provide. the rest of your statement provides one evolutionary mechanism by which gibbons became good brachiators, not evidence that they’re currently not.

doodlebugging 5y ago

After reading this article I have decided that the researchers earlier conclusions about how the rats keep track of their environment is suffering from a serious defect introduced by the researchers at the start. Their earlier work involved studying rat and bat navigation in unnatural environments - a "2D" space which was really a 3D space constrained to be along a single level. This forced or even allowed the rat brains to dumb things down to a situation where they had no need or opportunity to consider things outside that unnaturally limited space and so their brains naturally optimized for that simplistic situation. You have dumbed down their environment to the point where you have established a minimal ability that they need to master in order to function and navigate that environment. Then when you discover that their memory encodes navigational information from this dumbed-down, unnatural environment in a regularized grid, this should be no surprise. They are minimizing the energy required to thrive in that environment. Once you allow them to navigate in a more natural environment you should expect to see this regular grid disappear since their options for reaching a destination are no longer constrained to a single path along most of the route to their destination. Their brains will have to incorporate many more clues to guide their route selection and those clues could be time-varying as in the case where they are tracking the source of an odor as they search for food so they will need to monitor air flow and direction, odor intensity, potential obstacles between themselves and the source, alternate routes which present themselves as they move through space, threat detection from predators or bait traps, etc. In short, there so many variables that will need to be evaluated once you remove the flat plane constraints that led to their grid discovery that it should be no surprise to discover that a more complex environment uses different optimizations, some of which are encoded in a regularized notation and others, probably situational events on the path-picking decision tree, are more random.

bckr 5y ago
I think it's important to see all of these situations. The very fact that the brain optimizes its maps based on constrained environments is awesome. Also, the technology had to be built to do the previous 2d studies, which then bootstrapped the 3d studies. Knowing that under certain circumstances the neural maps had a very "clean architecture" may have saved a lot of time and effort once these more random 3d structures were being analyzed.