4 ms·

The issue with your question is that the dimensions of the configuration space and the physical form aren’t even approximately the same thing. Take, for example

by riwsky 6y ago

The issue with your question is that the dimensions of the configuration space and the physical form aren’t even approximately the same thing. Take, for example, a 100x100 grayscale image. It’s a flat image; the physical dimensions are 2. There are 10,000 different pixels though, and they are all allowed to vary independently of each other; the configuration-space dimensions are 10,000. Neurons are like the pixels in this analogy, not like the the physical dimensions.

posterboy 6y ago
Neurons are not random access. The analogy is otherwise pretty apt, except that an image doesn't store information about what it displays and I don't mean EXIF.
mapt 6y ago
Neurons aren't allowed to vary independently of each other, and neither are pixels; A grayscale image with random pixels is static, not even recognizable as an image. The mind cannot decode those pixels in a seven-dimensional indexing scheme, it can't even decode them in the given two dimensions if you have an array size error and store the same data in an array 87 columns wide. In your analogy, if you put a stop sign into the upper right side of the image, that is always going to be recalled associativity with the green caterpillar you put in the lower left side of the image. These properties don't work so well for memories & imperfect/error-prone but statistically correct biological systems. The average neuron has 1000 synapses, and for geometric reasons (Synaptic connections take up space) most of those are to other neurons that aren't very far away in 3D space.
- riwsky 6y ago
  You’re not contradicting my point, you’re just using the word “image” to refer to a different concept than I did. That “100x100 pictures of the real world” won’t reflect all the 10,000 dimensions available in “a 100x100 bitmap” is certainly the case—and that is what is meant by saying they lie on a lower-dimensional manifold within that space. Similarly: yes, physics limits neuronal connectivity. The actual space of neuronal connections lies on a manifold inside the full “n squared, divided by 2” dimensions of connectivity of any old set of n points. That still doesn’t mean neurons can’t represent high-dimensional concepts, because your treatment of physical dimensions as the same thing as concept space is still mistaken. Taking your 1000 synapses number for granted, the input to a given neuron would be 1000-dimensional, not three. If you’re not arguing the concept space is 3d, and merely arguing against those who’d say neuronal connectivity isn’t limited by physical constraints, then I’d advise a reread of the ancestor comments; none of them are saying that.

by riwsky 6y ago

posterboy 6y ago

Neurons are not random access. The analogy is otherwise pretty apt, except that an image doesn't store information about what it displays and I don't mean EXIF.

mapt 6y ago

Neurons aren't allowed to vary independently of each other, and neither are pixels; A grayscale image with random pixels is static, not even recognizable as an image. The mind cannot decode those pixels in a seven-dimensional indexing scheme, it can't even decode them in the given two dimensions if you have an array size error and store the same data in an array 87 columns wide. In your analogy, if you put a stop sign into the upper right side of the image, that is always going to be recalled associativity with the green caterpillar you put in the lower left side of the image. These properties don't work so well for memories & imperfect/error-prone but statistically correct biological systems. The average neuron has 1000 synapses, and for geometric reasons (Synaptic connections take up space) most of those are to other neurons that aren't very far away in 3D space.

riwsky 6y ago
You’re not contradicting my point, you’re just using the word “image” to refer to a different concept than I did. That “100x100 pictures of the real world” won’t reflect all the 10,000 dimensions available in “a 100x100 bitmap” is certainly the case—and that is what is meant by saying they lie on a lower-dimensional manifold within that space. Similarly: yes, physics limits neuronal connectivity. The actual space of neuronal connections lies on a manifold inside the full “n squared, divided by 2” dimensions of connectivity of any old set of n points. That still doesn’t mean neurons can’t represent high-dimensional concepts, because your treatment of physical dimensions as the same thing as concept space is still mistaken. Taking your 1000 synapses number for granted, the input to a given neuron would be 1000-dimensional, not three. If you’re not arguing the concept space is 3d, and merely arguing against those who’d say neuronal connectivity isn’t limited by physical constraints, then I’d advise a reread of the ancestor comments; none of them are saying that.