4 ms·

Could you elaborate why an epiphenomenon isn't viable? Being based on grandmother cell theory seems like an advantage.

jamesblonde 8y ago
In the grandmother cell theory people said - hey this cell only seems to fire when i show you a picture of your grandmother. Hence all the hierarchical features that make up your mental representation of your granny are found in the extraordinary binding properties of this one neuron. Of course, if you know large sparse DNNs, you know that you have 50 input vectors and pick any neuron far from the input, it may only activate in response to one of the 50 input vectors. It doesn't mean that it is the input vector. Similarly, Jeff has a theory that all representations in the brain include spatial representations. Now he claims the grid cell does that. But how would the grid cell have evolved to do provide the spatial coordinates? Could not the spatial coordinates be part of the sparse coding for the entity in question - why do they need the grid cells?
- lucidrains 8y ago
  I think you misunderstood the paper. Grid cells don't encode location, a set of grid cell modules do. So there isn't a grandmother cell here at all. He's simply proposing this theory because of the noted presence of grid cells in the neocortex. His new theory, if I understand it correctly, is pretty basic. It is simply saying every sparse encoding of sensory inputs is also tied to a spatial encoding by sets of grid cell modules within the same cortical column. For deep learning practitioners, it would be like saying every embedding comes with it's own spatial encoding, or placed into the same embedding space, however you want to look at it. These spatial encodings are then related to other spatial encodings of other sensory inputs on different cortical column stacks (by displacement cells?), to firstly build up representations of objects, and then, he surmises further, the relationships of these objects with each other (part to whole). I think the last part is probably what Jeff will need to substantiate a bit more.
  - monocasa 8y ago
    Wait, I thought grid cells were hippocampal cells, and therefore not in cortical columns.
    - lucidrains 8y ago
      Yea, you are right. That's the current thought. I think Numenta is basing their claim on other studies that suggest there are grid cell like signatures in the neocortex, thus leading to the rampant speculation.
    - SubiculumCode 8y ago
      place cells hippocampus grid cell entorhinal cortex is the usual way its put
- deleted 8y ago
  [deleted]

In the grandmother cell theory people said - hey this cell only seems to fire when i show you a picture of your grandmother. Hence all the hierarchical features that make up your mental representation of your granny are found in the extraordinary binding properties of this one neuron. Of course, if you know large sparse DNNs, you know that you have 50 input vectors and pick any neuron far from the input, it may only activate in response to one of the 50 input vectors. It doesn't mean that it is the input vector. Similarly, Jeff has a theory that all representations in the brain include spatial representations. Now he claims the grid cell does that. But how would the grid cell have evolved to do provide the spatial coordinates? Could not the spatial coordinates be part of the sparse coding for the entity in question - why do they need the grid cells?

lucidrains 8y ago
I think you misunderstood the paper. Grid cells don't encode location, a set of grid cell modules do. So there isn't a grandmother cell here at all. He's simply proposing this theory because of the noted presence of grid cells in the neocortex. His new theory, if I understand it correctly, is pretty basic. It is simply saying every sparse encoding of sensory inputs is also tied to a spatial encoding by sets of grid cell modules within the same cortical column. For deep learning practitioners, it would be like saying every embedding comes with it's own spatial encoding, or placed into the same embedding space, however you want to look at it. These spatial encodings are then related to other spatial encodings of other sensory inputs on different cortical column stacks (by displacement cells?), to firstly build up representations of objects, and then, he surmises further, the relationships of these objects with each other (part to whole). I think the last part is probably what Jeff will need to substantiate a bit more.
- monocasa 8y ago
  Wait, I thought grid cells were hippocampal cells, and therefore not in cortical columns.
  - lucidrains 8y ago
    Yea, you are right. That's the current thought. I think Numenta is basing their claim on other studies that suggest there are grid cell like signatures in the neocortex, thus leading to the rampant speculation.
  - SubiculumCode 8y ago
    place cells hippocampus grid cell entorhinal cortex is the usual way its put
deleted 8y ago
[deleted]