9 ms·

The brain's 5-million core, 9 Hz computer

201 points by liuhenry 15y ago

asadotzler 15y ago
nature's had a long time to sort out some decent hardware and software configurations. let's follow with computers.
3pt14159 15y ago
At the time of this comment this article has 35 points and 0 comments. Over the past 6 months or so I've noticed a trend that the best HN worthy articles often have points : comments ratios of 5 to 1 or less. This is a clear indication of that.
- burgerbrain 15y ago
  Maybe most of use read the articles.
- ChrisMac 15y ago
  For what it's worth, in the case of this article I found it very interesting, but since I only have a passing knowledge of the topics it's discussing, I don't feel there's anything I can personally contribute to a conversation about it. It's still a great article though, and a lack of comments doesn't detract from that.
- guscost 15y ago
  People tend to read interesting content, and reply to sensational/controversial content. The two categories are often mutually exclusive.
forkandwait 15y ago
When we finally design a simulated brain, I wonder if (1) it will be really good at spatial and behavioral behavior (balancing, motor skills, quick non-explicit decision making etc), but really bad at doing math? Not to say we shouldn't keep trying, but we all seem to think that the best computer will evolve from solving lots and lots of partial differential equations into being like the brain, but animal brains have evolved to solve really different questions than why we have been making computers.
- dkersten 15y ago
  Thats ok, we can then connect a more traditional coprocessor to offload the math heavy work to (perhaps some kind of SIMD stream processor).
- shantanubala 15y ago
  Not to mention neuroplasticity: the brain literally can reallocate neurons to different functions based on whether they're used (and what functions are needed). It's a really fascinating aspect of psychological therapy and rehabilitation for people with birth defects or traumatic brain injuries. If a person has a damaged or missing part of their brain, they can develop the same functionality elsewhere. The brain is built to adapt to information instead of just processing information. EDIT: http://en.wikipedia.org/wiki/Neuroplasticity http://en.wikipedia.org/wiki/Neuroplasticity
- 27182818284 15y ago
  On Intelligence is a good, novice introduction to the brain, its parts like the neocortex, and how studying the brain can help us design better artificial brains. One point the book made that I thought was very insightful was that once we understand brains well enough to make artificial ones, we can pick and choose features like a little bit of Einstein mixed with a little bit of Zuckerberg's brain. Anyway, if you have some spare time, check it out. It is a short book and I learned a few things.
- bermanoid 15y ago
  Not to say we shouldn't keep trying, but we all seem to think that the best computer will evolve from solving lots and lots of partial differential equations into being like the brain, but animal brains have evolved to solve really different questions than why we have been making computers. The thing that always strikes me as interesting about the evolution of intelligence is that it's a mostly unnecessary latecomer to the evolutionary party: plenty of things thrive in nature without anything even remotely resembling human intelligence, even plenty of large animals that function quite well in complex environments rely mostly on hard coded "unintelligent" systems in their brains. The fact is, most of the problems that need to be solved to survive and reproduce can be solved very effectively without much general intelligence. Sure, once evolution discovers intelligence it turns out to be a very efficient way to implement a lot of functionality that would otherwise need to be hard coded, but it's not strictly necessary, and nature did just fine without it for a long time. The good news: that also means that it's unlikely that significant evolutionary pressures went into designing the complex systems responsible for intelligent thought, so they were probably accidentally "designed" via random drift. If you think about that, given the complexity of the algorithms involved, it means that the algorithm space at that level complexity is probably fairly dense with algorithms that function intelligently (where by "fairly dense" I mean that it's a lot denser than we might assume otherwise, and the solution that evolution came up with is probably not the only possible one). That means if we ever figure out how to do a guided search through algorithms of the right complexity in the right way, we may yet stand a chance of "accidentally" discovering intelligence rather than deliberately designing it.
archgoon 15y ago
"Unlike transistors, neurons are intrinsically rhythmic to various degrees due to their ion channel complements that govern firing and refractory/recovery times. So external "clocking" is not always needed to make them run." Transistors don't need a clock in order to run. They can in fact be set up to create their own clocks. The purpose of clocks is for synchronization across the chip so that we mere mortals can modularize the operation of a CPU. That is, clocks exist mostly so that we can think in terms of sequential gate operations (or from the programmer point of view, assembly code). The author seems to confuse the chosen approach to designing computers (VLSI), and the actual physical capabilities of a transistor. We have opted over the last forty years to develop the CMOS logic gate way of organizing computers. There are other ways, as the brain demonstrates, but it is not clear at all that you can't do it with novel transistor topologies.
- brudgers 15y ago
  Whenever I see comparisons of the brain to a computer along with the implicit suggestion that computers will someday replicate brains, I am reminded of this passage from Leibniz's Monadology and that today's computers are but yesterday's mills: "Moreover, it must be confessed that perception and that which depends upon it are inexplicable on mechanical grounds, that is to say, by means of figures and motions. And supposing there were a machine, so constructed as to think, feel, and have perception, it might be conceived as increased in size, while keeping the same proportions, so that one might go into it as into a mill. That being so, we should, on examining its interior, find only parts which work one upon another, and never anything by which to explain a perception. Thus it is in a simple substance, and not in a compound or in a machine, that perception must be sought for. Further, nothing but this (namely, perceptions and their changes) can be found in a simple substance. It is also in this alone that all the internal activities of simple substances can consist." [http://philosophy.eserver.org/leibniz-monadology.txt http://philosophy.eserver.org/leibniz-monadology.txt] [http://en.wikipedia.org/wiki/Monadology http://en.wikipedia.org/wiki/Monadology]
  - archgoon 15y ago
    Wow. Leibniz constructed the Chinese Room Argument nearly four hundred years before John Searle or computers. Nice. :)
programminggeek 15y ago
The brain is also water cooled. Without proper water cooling it overheats causing segfaults and a white screen of death.
- ignifero 15y ago
  i disagree on the screen color
1337p337 15y ago
This reminds me of Chuck Moore's 144-core, unclocked (!) colorForth CPU: http://greenarraychips.com/home/documents/greg/GA144.htm http://greenarraychips.com/home/documents/greg/GA144.htm I wonder when he'll catch up to 5 million.
- Jach 15y ago
  http://colorforth.com/blog.htm http://colorforth.com/blog.htm "Here's GreenArrays' latest receipt of GA144 chips: 12 wafers; 14561 chips; 2,096,784 computers. And the fun is just beginning."
guscost 15y ago
I've been fascinated by Buzsaki and others in complementary fields since learning of them, and I've also written down some of my own ideas on the subject, from a purely theoretical perspective at least. I can't wait for what we might see in the next few decades. http://guscost.com/2011/04/12/science-analog-confabulation/ http://guscost.com/2011/04/12/science-analog-confabulation/
lallysingh 15y ago
Hmm, so, if their metaphor really held, the brain's computation could be simulated with a 45MHz CPU? Well, let's fix this up a bit... (1) Give it 1000 clock cycles of cpu work to simulate a single neuro-tick. (2) The clock is actually variable 5-500Hz (from the article). So, 500Hz*5m = 2.5GHz, at 1000 cycles required, so 2500 Ghz of CPU power. An amazon large-instance cluster box is 8 cores of xeon at 2.93 Ghz, about 110 cluster instances to simulate a brain?
- deleted 15y ago
  [deleted]
- eru 15y ago
  There's a bet over on longbets that in hundred years time (or so) somebody will have found a way to run AI on computers from around now. I'd bet that's probably actually doable, but I believe that the first version of AI will need more hardware than that. Just because we don't know what we are doing.
- IvoDankolov 15y ago
  Bare in mind that you also need to somehow do a brain scan with enough resolution to map every single connection if you want something functional. Also, what are you going to do about the sensory inputs (sight, hearing, nervous system). I'm sure it can't be good for a person's sanity to suddenly get completely 100% disconnected from the world. And while a 1:1 map of the brain would maybe be feasible to do at the moment or in a few years, but very expensive (although getting ever cheaper), it's kind of like the straw-man AI researcher's answer to everything : do a neural network, repeat training set and hope that it computes what we want. Trying to flaunt your lack of knowledge and trying clever ways to avoid solving the problem yourself is just asking for trouble in so many ways. In the end, the point is that AI is a software problem. Adding hardware will allow you a more stupid approach in solving it, but effective solutions are those that matter most.
  - robryan 15y ago
    Imitating the brain may be a matter of having enough computation power to allow a simple method to work. It reminds me of NLP where it seems we are doing a full circle back to the simple methods like Naive Bayes as they are showing better results than complex methods with enough data.
ignifero 15y ago
While it's true that some cells, e.g. pyramidals in the hippocampus can exhibit intrinsic oscillations, it's not true for most of the brain. Plus rhythms usually arise in networks, not single cells, and require the network to sustain themselves (that's why for example theta doesn't persist in vitro). Is there an example of a cell that can generate rhythms?
- cyrus_ 15y ago
  You are correct, single neurons are not intrinsic oscillators in vivo as far as anyone can tell. They could theoretically be -- there are ion channel combinations that would put neurons into an intrinsic oscillatory regime -- but that isn't really seen afaik. Oscillations in the brain arise due to excitatory-inhibitory loops.
CountHackulus 15y ago
Just a small nitpick. While the state of the art in x86 land might be 3GHz, the IBM Power chips and I think the Z mainframes too, not sure on that, have gone far beyond that speed. POWER6 chips reached 5.0GHz in 2008: http://en.wikipedia.org/wiki/POWER6 http://en.wikipedia.org/wiki/POWER6 POWER7 chips however have been clocked down to 4.25GHz: http://en.wikipedia.org/wiki/POWER7 http://en.wikipedia.org/wiki/POWER7
- Symmetry 15y ago
  The limit to how fast you can clock a CPU is mostly due to how long the chains of CMOS devices run before they hit a cocked latch. Having one CMOS drive 10 other devices takes longer than having one CMOS drive only 1, so there's a standard metric called the FO4 (Fan Out of 4) that can be used to measure gate delay. IBM has often put more effort into having low fo4s for its processors (usually around 15 or) vs. x86 (around 20).
vl 15y ago
My layman thinking on related subject of building brain-like computer is that currently it's possible (although expensive) to build required hardware - i.e. custom system that will have required number of connected electronic neurons, but I don't see a way to "boot it up". Human brains essentially boots from the DNA - i.e. it's layout and basic functions are predetermined by the DNA and then it gets trained for specifics. Even if we can train machine brain, how do we boot it up to the trainable state?
arapidhs 15y ago
transisotr technology cannot emulate brain activity. something new and more analog oriented suits better imho
SeamusBrady 15y ago
Some of the comments seem to revolve around the confusion of maps with territory - CF Lewis Carroll's map "the scale of a mile to the mile" - http://en.wikipedia.org/wiki/Map%E2%80%93territory_relation#.22The_map_is_not_the_territory.22 http://en.wikipedia.org/wiki/Map%E2%80%93territory_relation#...

201 points by liuhenry 15y ago

nature's had a long time to sort out some decent hardware and software configurations. let's follow with computers.

At the time of this comment this article has 35 points and 0 comments. Over the past 6 months or so I've noticed a trend that the best HN worthy articles often have points : comments ratios of 5 to 1 or less. This is a clear indication of that.

burgerbrain 15y ago
Maybe most of use read the articles.
ChrisMac 15y ago
For what it's worth, in the case of this article I found it very interesting, but since I only have a passing knowledge of the topics it's discussing, I don't feel there's anything I can personally contribute to a conversation about it. It's still a great article though, and a lack of comments doesn't detract from that.
guscost 15y ago
People tend to read interesting content, and reply to sensational/controversial content. The two categories are often mutually exclusive.

forkandwait 15y ago

When we finally design a simulated brain, I wonder if (1) it will be really good at spatial and behavioral behavior (balancing, motor skills, quick non-explicit decision making etc), but really bad at doing math? Not to say we shouldn't keep trying, but we all seem to think that the best computer will evolve from solving lots and lots of partial differential equations into being like the brain, but animal brains have evolved to solve really different questions than why we have been making computers.

dkersten 15y ago
Thats ok, we can then connect a more traditional coprocessor to offload the math heavy work to (perhaps some kind of SIMD stream processor).
shantanubala 15y ago
Not to mention neuroplasticity: the brain literally can reallocate neurons to different functions based on whether they're used (and what functions are needed). It's a really fascinating aspect of psychological therapy and rehabilitation for people with birth defects or traumatic brain injuries. If a person has a damaged or missing part of their brain, they can develop the same functionality elsewhere. The brain is built to adapt to information instead of just processing information. EDIT: http://en.wikipedia.org/wiki/Neuroplasticity http://en.wikipedia.org/wiki/Neuroplasticity
27182818284 15y ago
On Intelligence is a good, novice introduction to the brain, its parts like the neocortex, and how studying the brain can help us design better artificial brains. One point the book made that I thought was very insightful was that once we understand brains well enough to make artificial ones, we can pick and choose features like a little bit of Einstein mixed with a little bit of Zuckerberg's brain. Anyway, if you have some spare time, check it out. It is a short book and I learned a few things.
bermanoid 15y ago
Not to say we shouldn't keep trying, but we all seem to think that the best computer will evolve from solving lots and lots of partial differential equations into being like the brain, but animal brains have evolved to solve really different questions than why we have been making computers. The thing that always strikes me as interesting about the evolution of intelligence is that it's a mostly unnecessary latecomer to the evolutionary party: plenty of things thrive in nature without anything even remotely resembling human intelligence, even plenty of large animals that function quite well in complex environments rely mostly on hard coded "unintelligent" systems in their brains. The fact is, most of the problems that need to be solved to survive and reproduce can be solved very effectively without much general intelligence. Sure, once evolution discovers intelligence it turns out to be a very efficient way to implement a lot of functionality that would otherwise need to be hard coded, but it's not strictly necessary, and nature did just fine without it for a long time. The good news: that also means that it's unlikely that significant evolutionary pressures went into designing the complex systems responsible for intelligent thought, so they were probably accidentally "designed" via random drift. If you think about that, given the complexity of the algorithms involved, it means that the algorithm space at that level complexity is probably fairly dense with algorithms that function intelligently (where by "fairly dense" I mean that it's a lot denser than we might assume otherwise, and the solution that evolution came up with is probably not the only possible one). That means if we ever figure out how to do a guided search through algorithms of the right complexity in the right way, we may yet stand a chance of "accidentally" discovering intelligence rather than deliberately designing it.

archgoon 15y ago

"Unlike transistors, neurons are intrinsically rhythmic to various degrees due to their ion channel complements that govern firing and refractory/recovery times. So external "clocking" is not always needed to make them run." Transistors don't need a clock in order to run. They can in fact be set up to create their own clocks. The purpose of clocks is for synchronization across the chip so that we mere mortals can modularize the operation of a CPU. That is, clocks exist mostly so that we can think in terms of sequential gate operations (or from the programmer point of view, assembly code). The author seems to confuse the chosen approach to designing computers (VLSI), and the actual physical capabilities of a transistor. We have opted over the last forty years to develop the CMOS logic gate way of organizing computers. There are other ways, as the brain demonstrates, but it is not clear at all that you can't do it with novel transistor topologies.

brudgers 15y ago
Whenever I see comparisons of the brain to a computer along with the implicit suggestion that computers will someday replicate brains, I am reminded of this passage from Leibniz's Monadology and that today's computers are but yesterday's mills: "Moreover, it must be confessed that perception and that which depends upon it are inexplicable on mechanical grounds, that is to say, by means of figures and motions. And supposing there were a machine, so constructed as to think, feel, and have perception, it might be conceived as increased in size, while keeping the same proportions, so that one might go into it as into a mill. That being so, we should, on examining its interior, find only parts which work one upon another, and never anything by which to explain a perception. Thus it is in a simple substance, and not in a compound or in a machine, that perception must be sought for. Further, nothing but this (namely, perceptions and their changes) can be found in a simple substance. It is also in this alone that all the internal activities of simple substances can consist." [http://philosophy.eserver.org/leibniz-monadology.txt http://philosophy.eserver.org/leibniz-monadology.txt] [http://en.wikipedia.org/wiki/Monadology http://en.wikipedia.org/wiki/Monadology]
- archgoon 15y ago
  Wow. Leibniz constructed the Chinese Room Argument nearly four hundred years before John Searle or computers. Nice. :)

programminggeek 15y ago

The brain is also water cooled. Without proper water cooling it overheats causing segfaults and a white screen of death.

ignifero 15y ago
i disagree on the screen color

1337p337 15y ago

This reminds me of Chuck Moore's 144-core, unclocked (!) colorForth CPU: http://greenarraychips.com/home/documents/greg/GA144.htm http://greenarraychips.com/home/documents/greg/GA144.htm I wonder when he'll catch up to 5 million.

Jach 15y ago
http://colorforth.com/blog.htm http://colorforth.com/blog.htm "Here's GreenArrays' latest receipt of GA144 chips: 12 wafers; 14561 chips; 2,096,784 computers. And the fun is just beginning."

guscost 15y ago

I've been fascinated by Buzsaki and others in complementary fields since learning of them, and I've also written down some of my own ideas on the subject, from a purely theoretical perspective at least. I can't wait for what we might see in the next few decades. http://guscost.com/2011/04/12/science-analog-confabulation/ http://guscost.com/2011/04/12/science-analog-confabulation/

lallysingh 15y ago

Hmm, so, if their metaphor really held, the brain's computation could be simulated with a 45MHz CPU? Well, let's fix this up a bit... (1) Give it 1000 clock cycles of cpu work to simulate a single neuro-tick. (2) The clock is actually variable 5-500Hz (from the article). So, 500Hz*5m = 2.5GHz, at 1000 cycles required, so 2500 Ghz of CPU power. An amazon large-instance cluster box is 8 cores of xeon at 2.93 Ghz, about 110 cluster instances to simulate a brain?

deleted 15y ago
[deleted]
eru 15y ago
There's a bet over on longbets that in hundred years time (or so) somebody will have found a way to run AI on computers from around now. I'd bet that's probably actually doable, but I believe that the first version of AI will need more hardware than that. Just because we don't know what we are doing.
IvoDankolov 15y ago
Bare in mind that you also need to somehow do a brain scan with enough resolution to map every single connection if you want something functional. Also, what are you going to do about the sensory inputs (sight, hearing, nervous system). I'm sure it can't be good for a person's sanity to suddenly get completely 100% disconnected from the world. And while a 1:1 map of the brain would maybe be feasible to do at the moment or in a few years, but very expensive (although getting ever cheaper), it's kind of like the straw-man AI researcher's answer to everything : do a neural network, repeat training set and hope that it computes what we want. Trying to flaunt your lack of knowledge and trying clever ways to avoid solving the problem yourself is just asking for trouble in so many ways. In the end, the point is that AI is a software problem. Adding hardware will allow you a more stupid approach in solving it, but effective solutions are those that matter most.
- robryan 15y ago
  Imitating the brain may be a matter of having enough computation power to allow a simple method to work. It reminds me of NLP where it seems we are doing a full circle back to the simple methods like Naive Bayes as they are showing better results than complex methods with enough data.

ignifero 15y ago

While it's true that some cells, e.g. pyramidals in the hippocampus can exhibit intrinsic oscillations, it's not true for most of the brain. Plus rhythms usually arise in networks, not single cells, and require the network to sustain themselves (that's why for example theta doesn't persist in vitro). Is there an example of a cell that can generate rhythms?

cyrus_ 15y ago
You are correct, single neurons are not intrinsic oscillators in vivo as far as anyone can tell. They could theoretically be -- there are ion channel combinations that would put neurons into an intrinsic oscillatory regime -- but that isn't really seen afaik. Oscillations in the brain arise due to excitatory-inhibitory loops.

CountHackulus 15y ago

Just a small nitpick. While the state of the art in x86 land might be 3GHz, the IBM Power chips and I think the Z mainframes too, not sure on that, have gone far beyond that speed. POWER6 chips reached 5.0GHz in 2008: http://en.wikipedia.org/wiki/POWER6 http://en.wikipedia.org/wiki/POWER6 POWER7 chips however have been clocked down to 4.25GHz: http://en.wikipedia.org/wiki/POWER7 http://en.wikipedia.org/wiki/POWER7

Symmetry 15y ago
The limit to how fast you can clock a CPU is mostly due to how long the chains of CMOS devices run before they hit a cocked latch. Having one CMOS drive 10 other devices takes longer than having one CMOS drive only 1, so there's a standard metric called the FO4 (Fan Out of 4) that can be used to measure gate delay. IBM has often put more effort into having low fo4s for its processors (usually around 15 or) vs. x86 (around 20).

vl 15y ago

My layman thinking on related subject of building brain-like computer is that currently it's possible (although expensive) to build required hardware - i.e. custom system that will have required number of connected electronic neurons, but I don't see a way to "boot it up". Human brains essentially boots from the DNA - i.e. it's layout and basic functions are predetermined by the DNA and then it gets trained for specifics. Even if we can train machine brain, how do we boot it up to the trainable state?

arapidhs 15y ago

transisotr technology cannot emulate brain activity. something new and more analog oriented suits better imho

SeamusBrady 15y ago

Some of the comments seem to revolve around the confusion of maps with territory - CF Lewis Carroll's map "the scale of a mile to the mile" - http://en.wikipedia.org/wiki/Map%E2%80%93territory_relation#.22The_map_is_not_the_territory.22 http://en.wikipedia.org/wiki/Map%E2%80%93territory_relation#...