3 ms·

Start by adjusting your estimates. If you assume just neurons and synapses, you're on the petaflop scale. The two first entries on the TOP500 list barely get th

by groby_b 10y ago

Start by adjusting your estimates. If you assume just neurons and synapses, you're on the petaflop scale. The two first entries on the TOP500 list barely get there. (92 & 38 petaflops, brain assumed at ~30) Supercomputing instances built w/ EC2 are in the 0.25Petaflop range. Assuming that's only a fraction of their machines, maybe the major cloud providers play on the same scale - and at this point, you have to deal with communication across DCs, which might make realtime simulation utterly infeasible. That's 2 yes, and 3-5 maybes, not "over 1000 organizations". Then you're looking at the fact that recent research indicates dendrites carry processing power as well, which would move any brain simulation well into the exascale range. And Moore's law is shakey by now. But given all that is overcome - we seem to be doing OK at the scaling up thing - you would need a complete connectome of the brain. So far, we've succeeded doing that for roundworms. (And 6502's, as the article shows :) The human brain is a far cry off from that. But let's be optimists, and stipulate that is at some point done as well. We're still in the dark, because we have no idea if a fully connected brain will work without being in a specific starting state. We're nowhere even close to map the state of a brain. And at this point, we're definitely in the realm of questions where we don't even know yet if we will find an answer at all. Want more curveballs? There's a non-zero probability that the brain can only be fully explained using QM explanations. You've just significantly shifted the scale of the simulation. I don't know NEVER is an appropriate word, but it's not totally obvious, either. Using the word "obvious" in this context (just like the word "never") means you've reduced the problem to pop-science, where everything is either black or white.

logicallee 10y ago
>If you assume just neurons and synapses, you're on the petaflop scale. Where are you getting this? Did you account for the fact that the average cortical neuron fires 0.16 times per second? [1] Put in terms of gigaherz, that is -- haha just kidding about gigaherz, that is 0.16 herz per neuron. Times 100 billion neurons. "Combined with the observation that 90% of neurons rarely fire" so you actually might not need to touch all 100 billion of them in a cycle. 100 billion * 0.16 gets you literally 16 gigahertz. This is four cores running at 4 gigaherz. Memory can be heavily compressed if necessary, without slipping out of the realtime budget. People don't seem to get that these are ANALOG SIGNALS. These neurons aren't etched in silicon. Simulating that is like simulating a mechanical watch or something - with 100 billion pieces. Your guess is ridiculously too-high. Do you know how long it takes for a signal to cross from one half of the brain to the other? 100 m/s is widely reported progation speed limit, and the average brain length is 167 mm[2] . That means you get 167 mm / 10000 mm = 1.67 milliseconds. That's long enough to cross a server room approximately 25,000 times. (I divided 20meters by C, getting 66 ns - the time at speed of light to cross a 20 meter room, and this value goes into 1.67 ms 25,032 times.) All of my numbers here are super-conservative, above-all the fact that I said only 1,000 server rooms were able to do this today. It's considerably higher. We're ahead of the computational budget required, not behind it. I simply disagree with your calculation. The rest of this paragraph is sarcastic: When mentioning that there's a non-zero chance that neurons use quantum mechanical effects (the way our etched silicon does, since gates are so small now that this has to be accounted for), you forget to also mention that there's a non-zero chance that souls are connected to brains using an as-yet unidentified organ and therefore nothing in the brain can be simulated given any amount of computational power. I mean, the chances of that aren't zero, right? [1] "Based on the energy budget of the brain, it appears that the average cortical neuron fires around 0.16 times per second" - http://aiimpacts.org/rate-of-neuron-firing/ http://aiimpacts.org/rate-of-neuron-firing/ [2] https://www.disabled-world.com/artman/publish/brain-facts.shtml https://www.disabled-world.com/artman/publish/brain-facts.sh...
- yk 10y ago
  Your numbers are absurdly optimistic. For starters, analog signals are a lot harder to simulate than digital circuits in silicon. The simplest fourth order Runge-Kutta needs 4 evaluations of the gradient plus 8 floating point operation per timestep per dimension and probably you can get away with something like an order of magnitude smaller timestep than the dynamics you want to simulate. That is a penalty of something like two order of magnitude plus the evaluation of the gradient. Second, the complexity does not lie in the neurons, there one gets away with something like 100 billion neurons * 100 Flops/ GHz (Prozessor) ~ 100 Prozessors per dimension if we completely neglect the gradient. But what we need is the dynamics of the connections. And based on that, the estimate of 100 PFlops simulation gives roughly 10^5 clock cycles per connection, which is not very much if you compare it to the latency of a standard database in the microsecond range. (Network latencies are in a similar order of magnitude.) About your argument, that it would be enough to only simulate firing neurons, well we have to simulate a neuron to detect if it is firing. For completeness, Kurzweil mentions in The Singularity is near much lower estimates for functional simulation of a brain in the TFLOPS range, however that is with optimized algorithms and we probably need a singularity first to get these optimized algorithms.
  - logicallee 10y ago
    I simply read the numbers different from how you do. This page I linked before , https://www.disabled-world.com/artman/publish/brain-facts.shtml https://www.disabled-world.com/artman/publish/brain-facts.sh... , says "The brain has a processing capacity of 0.1 quadrillion instructions per second" (though that phrasing is bizarre), saying that the cluster Roadrunner at 1.06 petaflops did 10x more computation. Roadrunner was built in 2008.
    - AstralStorm 10y ago
      That is a relatively low estimate. It has a few false assumptions: 1) Neurons are independent spatially and temporally. 2) Dendrites are not computationally active, again false. 3) State is not important. 4) Neurons are used to simulate a binary function and each only one.

by groby_b 10y ago

logicallee 10y ago

>If you assume just neurons and synapses, you're on the petaflop scale. Where are you getting this? Did you account for the fact that the average cortical neuron fires 0.16 times per second? [1] Put in terms of gigaherz, that is -- haha just kidding about gigaherz, that is 0.16 herz per neuron. Times 100 billion neurons. "Combined with the observation that 90% of neurons rarely fire" so you actually might not need to touch all 100 billion of them in a cycle. 100 billion * 0.16 gets you literally 16 gigahertz. This is four cores running at 4 gigaherz. Memory can be heavily compressed if necessary, without slipping out of the realtime budget. People don't seem to get that these are ANALOG SIGNALS. These neurons aren't etched in silicon. Simulating that is like simulating a mechanical watch or something - with 100 billion pieces. Your guess is ridiculously too-high. Do you know how long it takes for a signal to cross from one half of the brain to the other? 100 m/s is widely reported progation speed limit, and the average brain length is 167 mm[2] . That means you get 167 mm / 10000 mm = 1.67 milliseconds. That's long enough to cross a server room approximately 25,000 times. (I divided 20meters by C, getting 66 ns - the time at speed of light to cross a 20 meter room, and this value goes into 1.67 ms 25,032 times.) All of my numbers here are super-conservative, above-all the fact that I said only 1,000 server rooms were able to do this today. It's considerably higher. We're ahead of the computational budget required, not behind it. I simply disagree with your calculation. The rest of this paragraph is sarcastic: When mentioning that there's a non-zero chance that neurons use quantum mechanical effects (the way our etched silicon does, since gates are so small now that this has to be accounted for), you forget to also mention that there's a non-zero chance that souls are connected to brains using an as-yet unidentified organ and therefore nothing in the brain can be simulated given any amount of computational power. I mean, the chances of that aren't zero, right? [1] "Based on the energy budget of the brain, it appears that the average cortical neuron fires around 0.16 times per second" - http://aiimpacts.org/rate-of-neuron-firing/ http://aiimpacts.org/rate-of-neuron-firing/ [2] https://www.disabled-world.com/artman/publish/brain-facts.shtml https://www.disabled-world.com/artman/publish/brain-facts.sh...

yk 10y ago
Your numbers are absurdly optimistic. For starters, analog signals are a lot harder to simulate than digital circuits in silicon. The simplest fourth order Runge-Kutta needs 4 evaluations of the gradient plus 8 floating point operation per timestep per dimension and probably you can get away with something like an order of magnitude smaller timestep than the dynamics you want to simulate. That is a penalty of something like two order of magnitude plus the evaluation of the gradient. Second, the complexity does not lie in the neurons, there one gets away with something like 100 billion neurons * 100 Flops/ GHz (Prozessor) ~ 100 Prozessors per dimension if we completely neglect the gradient. But what we need is the dynamics of the connections. And based on that, the estimate of 100 PFlops simulation gives roughly 10^5 clock cycles per connection, which is not very much if you compare it to the latency of a standard database in the microsecond range. (Network latencies are in a similar order of magnitude.) About your argument, that it would be enough to only simulate firing neurons, well we have to simulate a neuron to detect if it is firing. For completeness, Kurzweil mentions in The Singularity is near much lower estimates for functional simulation of a brain in the TFLOPS range, however that is with optimized algorithms and we probably need a singularity first to get these optimized algorithms.
- logicallee 10y ago
  I simply read the numbers different from how you do. This page I linked before , https://www.disabled-world.com/artman/publish/brain-facts.shtml https://www.disabled-world.com/artman/publish/brain-facts.sh... , says "The brain has a processing capacity of 0.1 quadrillion instructions per second" (though that phrasing is bizarre), saying that the cluster Roadrunner at 1.06 petaflops did 10x more computation. Roadrunner was built in 2008.
  - AstralStorm 10y ago
    That is a relatively low estimate. It has a few false assumptions: 1) Neurons are independent spatially and temporally. 2) Dendrites are not computationally active, again false. 3) State is not important. 4) Neurons are used to simulate a binary function and each only one.