5 ms·

I think the point of the article is that brain neurons are not equivalent to the ML representation of neurons, so that count if 17 billion in the cortex would a

by jarfil 8y ago

I think the point of the article is that brain neurons are not equivalent to the ML representation of neurons, so that count if 17 billion in the cortex would actually require many more "ML neurons" to be simulated. It also explains how we can keep adding complexity even if no new neurons are being created, since the branches themselves act like extra neurons.

taneq 8y ago
Wouldn't that just mean the brain's more like a small-world network, with small highly connected 'blobs' tied together in a sparser, larger network?
- shagie 8y ago
  That is one theory. Multiple "agents" is something that has been proposed. I don't have the background to read beyond the abstract of https://eml.berkeley.edu/~webfac/malmendier/e218_sp06/Carrillo.pdf https://eml.berkeley.edu/~webfac/malmendier/e218_sp06/Carril... > We model the brain as a multi-agent organization. Based on recent neuroscience evidence, we assume that different systems of the brain have different time-horizons and different access to information. Introducing asymmetric information as a restriction on optimal choices generates endogenous constraints in decision-making. There's also Society of the Mind ( https://en.wikipedia.org/wiki/Society_of_Mind https://en.wikipedia.org/wiki/Society_of_Mind ) by Marvin Minsky (which is very readable) > A core tenet of Minsky's philosophy is that "minds are what brains do". The society of mind theory views the human mind and any other naturally evolved cognitive systems as a vast society of individually simple processes known as agents. These processes are the fundamental thinking entities from which minds are built, and together produce the many abilities we attribute to minds. The great power in viewing a mind as a society of agents, as opposed to the consequence of some basic principle or some simple formal system, is that different agents can be based on different types of processes with different purposes, ways of representing knowledge, and methods for producing results.
- mattkrause 8y ago
  The brain is a small world network[0], but article’s thesis is that a neuron (a single vertex in that small world) can actually compute quite a bit on its own—-it’s not an integrate-and-fire unit or a ReLU. I guess you could expand things out and treat each synapse, rather than each neuron, as as a unit, in which case there’s an extra level in your “world”. This is a little different from how people normally think of “connectivity” because the way information is routed within a cell is much more flexible than between them. Still, it’s not totally bonkers. [0] Just so we’re on the same page, when people talk about the brain as a small world network, the general idea is that a neuron in (say) auditory cortex mostly communicates with other auditory neurons, locally and in higher/lower auditory areas. It doesn’t talk to neurons representing (say) touch in the small of your back. Thus, there are far fewer than 86B! connections in the brain, but there are still an awful lot!
mycall 8y ago
Another point in the article is how the dendrites provide local storage and 2-layer NN processing for the neuron. Is this even considered in ML representations?

by jarfil 8y ago

taneq 8y ago

Wouldn't that just mean the brain's more like a small-world network, with small highly connected 'blobs' tied together in a sparser, larger network?

shagie 8y ago
That is one theory. Multiple "agents" is something that has been proposed. I don't have the background to read beyond the abstract of https://eml.berkeley.edu/~webfac/malmendier/e218_sp06/Carrillo.pdf https://eml.berkeley.edu/~webfac/malmendier/e218_sp06/Carril... > We model the brain as a multi-agent organization. Based on recent neuroscience evidence, we assume that different systems of the brain have different time-horizons and different access to information. Introducing asymmetric information as a restriction on optimal choices generates endogenous constraints in decision-making. There's also Society of the Mind ( https://en.wikipedia.org/wiki/Society_of_Mind https://en.wikipedia.org/wiki/Society_of_Mind ) by Marvin Minsky (which is very readable) > A core tenet of Minsky's philosophy is that "minds are what brains do". The society of mind theory views the human mind and any other naturally evolved cognitive systems as a vast society of individually simple processes known as agents. These processes are the fundamental thinking entities from which minds are built, and together produce the many abilities we attribute to minds. The great power in viewing a mind as a society of agents, as opposed to the consequence of some basic principle or some simple formal system, is that different agents can be based on different types of processes with different purposes, ways of representing knowledge, and methods for producing results.
mattkrause 8y ago
The brain is a small world network[0], but article’s thesis is that a neuron (a single vertex in that small world) can actually compute quite a bit on its own—-it’s not an integrate-and-fire unit or a ReLU. I guess you could expand things out and treat each synapse, rather than each neuron, as as a unit, in which case there’s an extra level in your “world”. This is a little different from how people normally think of “connectivity” because the way information is routed within a cell is much more flexible than between them. Still, it’s not totally bonkers. [0] Just so we’re on the same page, when people talk about the brain as a small world network, the general idea is that a neuron in (say) auditory cortex mostly communicates with other auditory neurons, locally and in higher/lower auditory areas. It doesn’t talk to neurons representing (say) touch in the small of your back. Thus, there are far fewer than 86B! connections in the brain, but there are still an awful lot!

mycall 8y ago

Another point in the article is how the dendrites provide local storage and 2-layer NN processing for the neuron. Is this even considered in ML representations?