7 ms·

How Does the Brain Represent the Objects We Touch?

49 points by laurex 8y ago

al_ramich 8y ago
Fascinating approach. To evolve the current AI methods more study is required on how the human brain works and doing studies on the very early stages of the human brain at its infancy has the most potential to move us forward. What comes after Deep Learning? Probably a version of biotech, a synthetic capability that can learn based on the external sensory input.
axilmar 8y ago
It would make sense that any information is stored as a key-value pair in the brain. In this case, the key is the sensory input, and the value is the type of surface that is being touched. Could the simplistic key-value pair work in creating more complex systems? perhaps it does, if keys are complex input structures which be divided into more keys in a tree-like fashion.
- pjc50 8y ago
  I don't think it's established that either "keys" or "values" exist in the brain. It's more "stimulus-response", where similar stimulus produces similar response. There's a nonlinear classifier going on which tends to create categories for us, but it's definitely possible to produce cross-category stimulus.
  - eli_gottlieb 8y ago
    There isn't a classifier in the brain, and there certainly isn't a key-value store. The brain mostly runs the sensorimotor systems based on top-down Bayesian prediction, not bottom-up classification or regression.
    - axilmar 8y ago
      The way living organisms behave certainly points towards a simple key-value store, where key is the current experience and value is the response.
      - eli_gottlieb 8y ago
        Scientifically speaking, no, that's not even remotely true. What on Earth gave you that impression?
mattbierner 8y ago
Interesting. My sense of touch has gotten messed up for week or so a few times and those experience really made me appreciate how the brain integrates the senses to create the feeling of "normal" existence. You don't even realize this until the system breaks down. The closest thing I compare my experiences to is: imagine interacting with the world using a pair of super awesome VR gloves. Not muted exactly, just somehow unreal feeling. Even though my muscles worked just fine, the differentness of sensation made it very to manipulate objects without concentration. This was especially true for fabrics and soft objects, which my brain found fascinating to play with but utterly incomprehensible. The unreal sensation slowly faded throughout the week but certain experiences—like trying to use pockets—would make the feeling of unreality come roaring back. I don't know if this fading was just my brain returning to normal or if it was my brain re-leaning and adjusting to what normal is.
- adwi 8y ago
  Thank you for sharing this. Is there a name for this condition? Did you receive a diagnosis?
- taneq 8y ago
  > The closest thing I compare my experiences to is: imagine interacting with the world using a pair of super awesome VR gloves. Not muted exactly, just somehow unreal feeling. Wow, this is really interesting because one of the side effects of excessive VR usage that I've read about is "hands not feeling real". Otherwise 'normal' people spend a ton of time in VR (where your hands are represented by largely immobile mitts at the location of your controllers) and start to feel like their actual hands 'aren't real'. If you don't mind saying, how did your perception of touch get affected in this way?
cromulent 8y ago
It's more complicated than that - other senses such as sight are also part of the representation. Blind people who regain their sight do not recognise cubes and spheres that they would recognise by touch. https://www.newyorker.com/tech/elements/people-cured-blindness-see https://www.newyorker.com/tech/elements/people-cured-blindne...
- escherplex 8y ago
  True. Also have to factor in the proprioceptive sense, EG, index finger in neurologically mapped space pressing on the arm of a chair. But what about the condition where an index finger is pressing against an adjacent thumb? Here percepts involve two conjoined sensors reporting as both sensors and objects being sensed. What's the priority source of the sensation?
  - AstralStorm 8y ago
    Why should there be any? Advanced neural networks can easily take in such ambiguous inputs. Heck, even contradictory, explaining a few illusions in the process.
ggm 8y ago
I think it's very unlikely we will ever concretely define how the brain represents reality in its intrior model. Given that I can conceptualize the existence of the letter "A" on the back face of a red square box sitting on a black floor and hypothetically view it face on,reflected in a mirror behind, through the box backwards if it's translucent, and the box doesn't even have to exist ... And then intrude being asked to put my hand into some three-space simulating feedback environment and "feel" the box which isn't there, or add VR glasses and "see" the box which isn't there... Now consider a classic nuclear hot box, and put a real cube inside it and interact through the remote sensor waldoes. Is that feeling any different? Is the brain model any different?

49 points by laurex 8y ago

Fascinating approach. To evolve the current AI methods more study is required on how the human brain works and doing studies on the very early stages of the human brain at its infancy has the most potential to move us forward. What comes after Deep Learning? Probably a version of biotech, a synthetic capability that can learn based on the external sensory input.

axilmar 8y ago

It would make sense that any information is stored as a key-value pair in the brain. In this case, the key is the sensory input, and the value is the type of surface that is being touched. Could the simplistic key-value pair work in creating more complex systems? perhaps it does, if keys are complex input structures which be divided into more keys in a tree-like fashion.

pjc50 8y ago
I don't think it's established that either "keys" or "values" exist in the brain. It's more "stimulus-response", where similar stimulus produces similar response. There's a nonlinear classifier going on which tends to create categories for us, but it's definitely possible to produce cross-category stimulus.
- eli_gottlieb 8y ago
  There isn't a classifier in the brain, and there certainly isn't a key-value store. The brain mostly runs the sensorimotor systems based on top-down Bayesian prediction, not bottom-up classification or regression.
  - axilmar 8y ago
    The way living organisms behave certainly points towards a simple key-value store, where key is the current experience and value is the response.
    - eli_gottlieb 8y ago
      Scientifically speaking, no, that's not even remotely true. What on Earth gave you that impression?

mattbierner 8y ago

Interesting. My sense of touch has gotten messed up for week or so a few times and those experience really made me appreciate how the brain integrates the senses to create the feeling of "normal" existence. You don't even realize this until the system breaks down. The closest thing I compare my experiences to is: imagine interacting with the world using a pair of super awesome VR gloves. Not muted exactly, just somehow unreal feeling. Even though my muscles worked just fine, the differentness of sensation made it very to manipulate objects without concentration. This was especially true for fabrics and soft objects, which my brain found fascinating to play with but utterly incomprehensible. The unreal sensation slowly faded throughout the week but certain experiences—like trying to use pockets—would make the feeling of unreality come roaring back. I don't know if this fading was just my brain returning to normal or if it was my brain re-leaning and adjusting to what normal is.

adwi 8y ago
Thank you for sharing this. Is there a name for this condition? Did you receive a diagnosis?
taneq 8y ago
> The closest thing I compare my experiences to is: imagine interacting with the world using a pair of super awesome VR gloves. Not muted exactly, just somehow unreal feeling. Wow, this is really interesting because one of the side effects of excessive VR usage that I've read about is "hands not feeling real". Otherwise 'normal' people spend a ton of time in VR (where your hands are represented by largely immobile mitts at the location of your controllers) and start to feel like their actual hands 'aren't real'. If you don't mind saying, how did your perception of touch get affected in this way?

cromulent 8y ago

It's more complicated than that - other senses such as sight are also part of the representation. Blind people who regain their sight do not recognise cubes and spheres that they would recognise by touch. https://www.newyorker.com/tech/elements/people-cured-blindness-see https://www.newyorker.com/tech/elements/people-cured-blindne...

escherplex 8y ago
True. Also have to factor in the proprioceptive sense, EG, index finger in neurologically mapped space pressing on the arm of a chair. But what about the condition where an index finger is pressing against an adjacent thumb? Here percepts involve two conjoined sensors reporting as both sensors and objects being sensed. What's the priority source of the sensation?
- AstralStorm 8y ago
  Why should there be any? Advanced neural networks can easily take in such ambiguous inputs. Heck, even contradictory, explaining a few illusions in the process.

ggm 8y ago

I think it's very unlikely we will ever concretely define how the brain represents reality in its intrior model. Given that I can conceptualize the existence of the letter "A" on the back face of a red square box sitting on a black floor and hypothetically view it face on,reflected in a mirror behind, through the box backwards if it's translucent, and the box doesn't even have to exist ... And then intrude being asked to put my hand into some three-space simulating feedback environment and "feel" the box which isn't there, or add VR glasses and "see" the box which isn't there... Now consider a classic nuclear hot box, and put a real cube inside it and interact through the remote sensor waldoes. Is that feeling any different? Is the brain model any different?