3 ms·

"With a million cores we only get to about 1% of the scale of the human brain" Interesting. So in terms of raw computing power, the totality of computers out t

by justin_ 11y ago

"With a million cores we only get to about 1% of the scale of the human brain" Interesting. So in terms of raw computing power, the totality of computers out there is much more powerful than a brain. I wonder when we first passed that level as a whole. (I know that we wouldn't use all our computers to make a "brain", but I think it's interesting to think about :)

bch 11y ago
> I know that we wouldn't use all our computers to make a "brain", but I think it's interesting to think about Instead of SETI@home [0], SAI@home (search for artificial intelligence) ? [0] setiathome.ssl.berkeley.edu/
etrautmann 11y ago
But the capabilities don't scale that way and the interconnect and topology of how that computation is arranged is completely different between brains and existing networks.
xixixao 11y ago
This is a very inaccurate comparison for many reasons, but to get the sense of complexity of the human brain: In the human brain, there are almost 100 billion neurons (the project plans to simulate 1000 neurons on one core, hence 1% atm). Each has about 7000 connections, or alternatively there are estimated 10^15 connections or more. The internet has only about 15-20 billion nodes, only few highly connected, the Internet is more of a tree than a complete mesh. So in some way, you could say we are nowhere close to obtaining the computing power of the brain, if we, of course, disregard the fact that the nodes, computers, are very powerful on their own. But this is where the comparison breaks. The human brain is powerful due to the complexity of its network, not the sheer amount of "working units" (cores/transistors). Comment on the project: it is exciting, but it must not be understated that it is still a very gross approximation, since the neuron models used are simple. It might be the case that interesting behavior will arise using this simple model, but it also might be the case that the secret sauce is in the fine behavior of each neuron. The hope is that interesting phenomena can be observed with this simpler model, perhaps a bit like classical physics can often be used without requiring the full model of quantum mechanics.
- cristianpascu 11y ago
  [Troll] 100 bn neurons * 7k connections + 1 soul. Nait! :)
- iandanforth 11y ago
  I would extend this to say that when you get down to the level of synapses you're now talking about memory and not CPU. (If you actually want to implement this) 100 trillion synapses is better mapped to 100TB of RAM. It's also very important to remember how incredibly slow the human brain is. Were talking 100s of ms slow to go from sensory input to motor output. A single neuron might take between 1 and 10 ms to fire, and a single dendrite might take 1/10th that time, so at best your doing computation at 10khz. CPU has 5 orders of magnitude over biology. The problem is we have only a vague idea of how the network is connected and don't really know the algorithm thats being implemented by that network. So we fall back on things like simulating ion channels which take way more compute resources than necessary. There is a lot of cargo culting going on right now, but of course it's also insanely exciting and fun to find out what does and doesn't work.
  - Asbostos 11y ago
    With 1000 neurons per core running in series, that would eat up a lot of the speed advantage.
Animats 11y ago
"With a million cores we only get to about 1% of the scale of the human brain" (or ten mouse brains), he says. This, he says, requires about 10 racks. So a human brain would require 1000 racks, or about half a typical Amazon data center, of which Amazon has more than a hundred. We'll have to see if he can do anything useful with all those CPUs.
kolbe 11y ago
> With a million cores we only get to about 1% of the scale of the human brain That statement also requires an assumption that there's a logical one-to-one mapping between an organic nueron and an ANN neuron. In reality, ANN's are modeled using very loose approximations of organic neurons. For all we know, within each organic neuron, there are billions of parallel 'threads' of 'computation' or there are links between neurons that a traditional model doesn't capture. Or any number of things. We're simply too far from understanding what actually happens in brains to even start talking about computational equivalency.

by justin_ 11y ago

bch 11y ago

> I know that we wouldn't use all our computers to make a "brain", but I think it's interesting to think about Instead of SETI@home [0], SAI@home (search for artificial intelligence) ? [0] setiathome.ssl.berkeley.edu/

etrautmann 11y ago

But the capabilities don't scale that way and the interconnect and topology of how that computation is arranged is completely different between brains and existing networks.

xixixao 11y ago

This is a very inaccurate comparison for many reasons, but to get the sense of complexity of the human brain: In the human brain, there are almost 100 billion neurons (the project plans to simulate 1000 neurons on one core, hence 1% atm). Each has about 7000 connections, or alternatively there are estimated 10^15 connections or more. The internet has only about 15-20 billion nodes, only few highly connected, the Internet is more of a tree than a complete mesh. So in some way, you could say we are nowhere close to obtaining the computing power of the brain, if we, of course, disregard the fact that the nodes, computers, are very powerful on their own. But this is where the comparison breaks. The human brain is powerful due to the complexity of its network, not the sheer amount of "working units" (cores/transistors). Comment on the project: it is exciting, but it must not be understated that it is still a very gross approximation, since the neuron models used are simple. It might be the case that interesting behavior will arise using this simple model, but it also might be the case that the secret sauce is in the fine behavior of each neuron. The hope is that interesting phenomena can be observed with this simpler model, perhaps a bit like classical physics can often be used without requiring the full model of quantum mechanics.

cristianpascu 11y ago
[Troll] 100 bn neurons * 7k connections + 1 soul. Nait! :)
iandanforth 11y ago
I would extend this to say that when you get down to the level of synapses you're now talking about memory and not CPU. (If you actually want to implement this) 100 trillion synapses is better mapped to 100TB of RAM. It's also very important to remember how incredibly slow the human brain is. Were talking 100s of ms slow to go from sensory input to motor output. A single neuron might take between 1 and 10 ms to fire, and a single dendrite might take 1/10th that time, so at best your doing computation at 10khz. CPU has 5 orders of magnitude over biology. The problem is we have only a vague idea of how the network is connected and don't really know the algorithm thats being implemented by that network. So we fall back on things like simulating ion channels which take way more compute resources than necessary. There is a lot of cargo culting going on right now, but of course it's also insanely exciting and fun to find out what does and doesn't work.
- Asbostos 11y ago
  With 1000 neurons per core running in series, that would eat up a lot of the speed advantage.

Animats 11y ago

"With a million cores we only get to about 1% of the scale of the human brain" (or ten mouse brains), he says. This, he says, requires about 10 racks. So a human brain would require 1000 racks, or about half a typical Amazon data center, of which Amazon has more than a hundred. We'll have to see if he can do anything useful with all those CPUs.

kolbe 11y ago

> With a million cores we only get to about 1% of the scale of the human brain That statement also requires an assumption that there's a logical one-to-one mapping between an organic nueron and an ANN neuron. In reality, ANN's are modeled using very loose approximations of organic neurons. For all we know, within each organic neuron, there are billions of parallel 'threads' of 'computation' or there are links between neurons that a traditional model doesn't capture. Or any number of things. We're simply too far from understanding what actually happens in brains to even start talking about computational equivalency.