4 ms·

It's about 100B neurons in a human brain [0], but neurons are also much slower. Individual neurons run at under 200Hz [1], which is probably a half dozen orders

It's about 100B neurons in a human brain [0], but neurons are also much slower. Individual neurons run at under 200Hz [1], which is probably a half dozen orders of magnitude lower than these processors are running at. Maybe it balances out? [0] https://www.verywellmind.com/how-many-neurons-are-in-the-brain-2794889 https://www.verywellmind.com/how-many-neurons-are-in-the-bra... [1] https://en.wikipedia.org/wiki/Neural_oscillation https://en.wikipedia.org/wiki/Neural_oscillation

Retric 8y ago
Neurons do a lot more than 1 calculation for each of those Hz. Dendritic branching can be extensive and in some cases is sufficient to receive as many as 100,000 inputs to a single neuron. https://en.wikipedia.org/wiki/Dendrite https://en.wikipedia.org/wiki/Dendrite A more conservative 10,000 * 200 Hz ~= 2 Mhz * 100B ~= 100 Million 2 Ghz processors cores. Though each dendrite is again doing far more than one calculation per each of those cycles.
- delecti 8y ago
  While true, the same is also true for many processor architectures. They're not just doing a single mathematical calculation per cycle, and additionally they also take multiple bits of input. A 32 bit processor could be thought of as having 32 "dendrites" of input, depending on what specific calculations are being performed to emulate the functions of neurons. I recognize that the specifics are very different between a single neuron and a single processor, but the processors are so much faster that I'm inclined to give some benefit of the doubt to the people actually in possession of the array of 1M processors.
  - yellowapple 8y ago
    So I guess by that analogy a neuron is equivalent to a rather slow processor with a 100-kilobit word size?
    - Retric 8y ago
      No, Dendrites are far more complex than bits. They have multiple chemical pathways and operate in real time. A you can think of a single dendrite as ~1,000 bits of information though it's hard to say how much of that information is useful as it's rather complex chemical signaling. Which gets into why opioids get people high for example.
  - deleted 8y ago
    [deleted]
marmaduke 8y ago
> Individual neurons run at under 200Hz [1] maybe but that doesn't account for extensive nonlinear computation happening along dendritic arbors. Think vast decision trees resolved in parallel, per neuron, at 200Hz.
amelius 8y ago
> Individual neurons run at under 200Hz So anything your brain can think in 1 second needs at most a depth of 200 layers.
- Retric 8y ago
  Each neuron operates on it's own clock cycle which messes with these calculations.
- jakelarkin 8y ago
  anecdotelly every neuron has ~7000 synapses so 200 layers among 7000^200 neurons? (its probably a meaningless analogy)
dkfellows 8y ago
The processor cores in SpiNNaker run at 200MHz (yes, this is slow!), and can usually issue one instruction per CPU cycle (that's quite nice). However, the trick to getting lots of neurons in is in keeping the number of CPU cycles per synapse event down, since there's a lot more of those than there are neurons. Yes, bits of the synapse processing code are in assembly coded to waste not one cycle at all. It turns out to be vital to do that in order to keep the efficiency high (and that has many key knock on effects; the synaptic density is a critical parameter for overall model scaling). In any case, to say that neurons process information at 200Hz is wrong. Or rather it is not even wrong. The individual neurons don't really do very much, but the overall network does a lot and it isn't limited to 200Hz at all. It's just that it handles time in a totally different way to conventional computers...

Retric 8y ago

Neurons do a lot more than 1 calculation for each of those Hz. Dendritic branching can be extensive and in some cases is sufficient to receive as many as 100,000 inputs to a single neuron. https://en.wikipedia.org/wiki/Dendrite https://en.wikipedia.org/wiki/Dendrite A more conservative 10,000 * 200 Hz ~= 2 Mhz * 100B ~= 100 Million 2 Ghz processors cores. Though each dendrite is again doing far more than one calculation per each of those cycles.

delecti 8y ago
While true, the same is also true for many processor architectures. They're not just doing a single mathematical calculation per cycle, and additionally they also take multiple bits of input. A 32 bit processor could be thought of as having 32 "dendrites" of input, depending on what specific calculations are being performed to emulate the functions of neurons. I recognize that the specifics are very different between a single neuron and a single processor, but the processors are so much faster that I'm inclined to give some benefit of the doubt to the people actually in possession of the array of 1M processors.
- yellowapple 8y ago
  So I guess by that analogy a neuron is equivalent to a rather slow processor with a 100-kilobit word size?
  - Retric 8y ago
    No, Dendrites are far more complex than bits. They have multiple chemical pathways and operate in real time. A you can think of a single dendrite as ~1,000 bits of information though it's hard to say how much of that information is useful as it's rather complex chemical signaling. Which gets into why opioids get people high for example.
- deleted 8y ago
  [deleted]

marmaduke 8y ago

> Individual neurons run at under 200Hz [1] maybe but that doesn't account for extensive nonlinear computation happening along dendritic arbors. Think vast decision trees resolved in parallel, per neuron, at 200Hz.

amelius 8y ago

> Individual neurons run at under 200Hz So anything your brain can think in 1 second needs at most a depth of 200 layers.

Retric 8y ago
Each neuron operates on it's own clock cycle which messes with these calculations.
jakelarkin 8y ago
anecdotelly every neuron has ~7000 synapses so 200 layers among 7000^200 neurons? (its probably a meaningless analogy)

dkfellows 8y ago

The processor cores in SpiNNaker run at 200MHz (yes, this is slow!), and can usually issue one instruction per CPU cycle (that's quite nice). However, the trick to getting lots of neurons in is in keeping the number of CPU cycles per synapse event down, since there's a lot more of those than there are neurons. Yes, bits of the synapse processing code are in assembly coded to waste not one cycle at all. It turns out to be vital to do that in order to keep the efficiency high (and that has many key knock on effects; the synaptic density is a critical parameter for overall model scaling). In any case, to say that neurons process information at 200Hz is wrong. Or rather it is not even wrong. The individual neurons don't really do very much, but the overall network does a lot and it isn't limited to 200Hz at all. It's just that it handles time in a totally different way to conventional computers...