5 ms·

>If you assume just neurons and synapses, you're on the petaflop scale. Where are you getting this? Did you account for the fact that the average cortical neu

>If you assume just neurons and synapses, you're on the petaflop scale. Where are you getting this? Did you account for the fact that the average cortical neuron fires 0.16 times per second? [1] Put in terms of gigaherz, that is -- haha just kidding about gigaherz, that is 0.16 herz per neuron. Times 100 billion neurons. "Combined with the observation that 90% of neurons rarely fire" so you actually might not need to touch all 100 billion of them in a cycle. 100 billion * 0.16 gets you literally 16 gigahertz. This is four cores running at 4 gigaherz. Memory can be heavily compressed if necessary, without slipping out of the realtime budget. People don't seem to get that these are ANALOG SIGNALS. These neurons aren't etched in silicon. Simulating that is like simulating a mechanical watch or something - with 100 billion pieces. Your guess is ridiculously too-high. Do you know how long it takes for a signal to cross from one half of the brain to the other? 100 m/s is widely reported progation speed limit, and the average brain length is 167 mm[2] . That means you get 167 mm / 10000 mm = 1.67 milliseconds. That's long enough to cross a server room approximately 25,000 times. (I divided 20meters by C, getting 66 ns - the time at speed of light to cross a 20 meter room, and this value goes into 1.67 ms 25,032 times.) All of my numbers here are super-conservative, above-all the fact that I said only 1,000 server rooms were able to do this today. It's considerably higher. We're ahead of the computational budget required, not behind it. I simply disagree with your calculation. The rest of this paragraph is sarcastic: When mentioning that there's a non-zero chance that neurons use quantum mechanical effects (the way our etched silicon does, since gates are so small now that this has to be accounted for), you forget to also mention that there's a non-zero chance that souls are connected to brains using an as-yet unidentified organ and therefore nothing in the brain can be simulated given any amount of computational power. I mean, the chances of that aren't zero, right? [1] "Based on the energy budget of the brain, it appears that the average cortical neuron fires around 0.16 times per second" - http://aiimpacts.org/rate-of-neuron-firing/ http://aiimpacts.org/rate-of-neuron-firing/ [2] https://www.disabled-world.com/artman/publish/brain-facts.shtml https://www.disabled-world.com/artman/publish/brain-facts.sh...

yk 10y ago
Your numbers are absurdly optimistic. For starters, analog signals are a lot harder to simulate than digital circuits in silicon. The simplest fourth order Runge-Kutta needs 4 evaluations of the gradient plus 8 floating point operation per timestep per dimension and probably you can get away with something like an order of magnitude smaller timestep than the dynamics you want to simulate. That is a penalty of something like two order of magnitude plus the evaluation of the gradient. Second, the complexity does not lie in the neurons, there one gets away with something like 100 billion neurons * 100 Flops/ GHz (Prozessor) ~ 100 Prozessors per dimension if we completely neglect the gradient. But what we need is the dynamics of the connections. And based on that, the estimate of 100 PFlops simulation gives roughly 10^5 clock cycles per connection, which is not very much if you compare it to the latency of a standard database in the microsecond range. (Network latencies are in a similar order of magnitude.) About your argument, that it would be enough to only simulate firing neurons, well we have to simulate a neuron to detect if it is firing. For completeness, Kurzweil mentions in The Singularity is near much lower estimates for functional simulation of a brain in the TFLOPS range, however that is with optimized algorithms and we probably need a singularity first to get these optimized algorithms.
- logicallee 10y ago
  I simply read the numbers different from how you do. This page I linked before , https://www.disabled-world.com/artman/publish/brain-facts.shtml https://www.disabled-world.com/artman/publish/brain-facts.sh... , says "The brain has a processing capacity of 0.1 quadrillion instructions per second" (though that phrasing is bizarre), saying that the cluster Roadrunner at 1.06 petaflops did 10x more computation. Roadrunner was built in 2008.
  - AstralStorm 10y ago
    That is a relatively low estimate. It has a few false assumptions: 1) Neurons are independent spatially and temporally. 2) Dendrites are not computationally active, again false. 3) State is not important. 4) Neurons are used to simulate a binary function and each only one.
    - logicallee 10y ago
      You are right, but not right enough to be "theoretically impossible" territory. Click my name and see my other comments in this thread.
stochastician 10y ago
I would recommend taking the "0.16 times per second" number with a real grain of salt. That neuron is evaluating inputs from (oh, say) 1k-10k sources simultaneously, computing some analog function that we don't really understand, which involves membrane dynamics, past spiking history, the amount of post-synaptic intracellular calcium, etc. These little units are absolutely incredible, and we don't quite know yet what parts of this behavior are actually the "secret sauce" that makes it work. For me the most authoritative answer is what the Human Brain Project in europe is able to achieve: http://www.nature.com/news/fragment-of-rat-brain-simulated-in-supercomputer-1.18536 http://www.nature.com/news/fragment-of-rat-brain-simulated-i... 31k cells with 37M synapses. That's a start, but pretty far from a mouse brain (100M cells) or a human brain (80B? ). Also if Moore's law really is dead, this all gets much more depressing.
- logicallee 10y ago
  You don't go into my most important figure, which is 100 m/s speed limit: this means that we can simulate a brain in a server room of any reasonable size (10,000 computers are no problem), racing back and forth thousands of times without issue. That radically simplifies the task, and makes the 1k-10k sources much easier. You mention membrane dynamics, past spiking history, etc - but neural nets accomplish a lot without these things. A simplification surely works. You are not thinking about simulating a brain using a large cluster with a lot of back and forth. Why not? Don't you get how incredibly slow 100 m/s is? This is 7 orders of magnitude difference in speed. Silicon is seven orders of magnitude faster at signal propagation than the brain, which isn't electrical. To you this doesn't unlock vast possibilities for simulating the analog in real time? It just doesn't seem that permanently impossible, given the results you are already claiming are occurring. Also although I directly addressed you, I guess you don't have any answer as to why someone would choose a target date like "never" or "300 years" as opposed to some reasonable extrapolation of the results being achieved today.

yk 10y ago

Your numbers are absurdly optimistic. For starters, analog signals are a lot harder to simulate than digital circuits in silicon. The simplest fourth order Runge-Kutta needs 4 evaluations of the gradient plus 8 floating point operation per timestep per dimension and probably you can get away with something like an order of magnitude smaller timestep than the dynamics you want to simulate. That is a penalty of something like two order of magnitude plus the evaluation of the gradient. Second, the complexity does not lie in the neurons, there one gets away with something like 100 billion neurons * 100 Flops/ GHz (Prozessor) ~ 100 Prozessors per dimension if we completely neglect the gradient. But what we need is the dynamics of the connections. And based on that, the estimate of 100 PFlops simulation gives roughly 10^5 clock cycles per connection, which is not very much if you compare it to the latency of a standard database in the microsecond range. (Network latencies are in a similar order of magnitude.) About your argument, that it would be enough to only simulate firing neurons, well we have to simulate a neuron to detect if it is firing. For completeness, Kurzweil mentions in The Singularity is near much lower estimates for functional simulation of a brain in the TFLOPS range, however that is with optimized algorithms and we probably need a singularity first to get these optimized algorithms.

logicallee 10y ago
I simply read the numbers different from how you do. This page I linked before , https://www.disabled-world.com/artman/publish/brain-facts.shtml https://www.disabled-world.com/artman/publish/brain-facts.sh... , says "The brain has a processing capacity of 0.1 quadrillion instructions per second" (though that phrasing is bizarre), saying that the cluster Roadrunner at 1.06 petaflops did 10x more computation. Roadrunner was built in 2008.
- AstralStorm 10y ago
  That is a relatively low estimate. It has a few false assumptions: 1) Neurons are independent spatially and temporally. 2) Dendrites are not computationally active, again false. 3) State is not important. 4) Neurons are used to simulate a binary function and each only one.
  - logicallee 10y ago
    You are right, but not right enough to be "theoretically impossible" territory. Click my name and see my other comments in this thread.

stochastician 10y ago

I would recommend taking the "0.16 times per second" number with a real grain of salt. That neuron is evaluating inputs from (oh, say) 1k-10k sources simultaneously, computing some analog function that we don't really understand, which involves membrane dynamics, past spiking history, the amount of post-synaptic intracellular calcium, etc. These little units are absolutely incredible, and we don't quite know yet what parts of this behavior are actually the "secret sauce" that makes it work. For me the most authoritative answer is what the Human Brain Project in europe is able to achieve: http://www.nature.com/news/fragment-of-rat-brain-simulated-in-supercomputer-1.18536 http://www.nature.com/news/fragment-of-rat-brain-simulated-i... 31k cells with 37M synapses. That's a start, but pretty far from a mouse brain (100M cells) or a human brain (80B? ). Also if Moore's law really is dead, this all gets much more depressing.

logicallee 10y ago
You don't go into my most important figure, which is 100 m/s speed limit: this means that we can simulate a brain in a server room of any reasonable size (10,000 computers are no problem), racing back and forth thousands of times without issue. That radically simplifies the task, and makes the 1k-10k sources much easier. You mention membrane dynamics, past spiking history, etc - but neural nets accomplish a lot without these things. A simplification surely works. You are not thinking about simulating a brain using a large cluster with a lot of back and forth. Why not? Don't you get how incredibly slow 100 m/s is? This is 7 orders of magnitude difference in speed. Silicon is seven orders of magnitude faster at signal propagation than the brain, which isn't electrical. To you this doesn't unlock vast possibilities for simulating the analog in real time? It just doesn't seem that permanently impossible, given the results you are already claiming are occurring. Also although I directly addressed you, I guess you don't have any answer as to why someone would choose a target date like "never" or "300 years" as opposed to some reasonable extrapolation of the results being achieved today.