4 ms·

>They are incredibly slow and energy inefficient; Human brains are energy inefficient? Well, thats a first ;) "In 1990, the legendary Caltech engineer Carver

by wamatt 9y ago

>They are incredibly slow and energy inefficient; Human brains are energy inefficient? Well, thats a first ;) "In 1990, the legendary Caltech engineer Carver Mead correctly predicted that our present-day computers would use ten million times more energy for a single instruction than the brain uses for a synaptic activation." "Last March, AlphaGo, a program created by Google DeepMind, was able to beat a world-champion human player of Go, but only after it had trained on a database of thirty million moves, running on approximately a million watts. (Its opponent’s brain, by contrast, would have been about fifty thousand times more energy-thrifty, consuming twenty watts.)" [1] http://www.newyorker.com/tech/elements/a-computer-to-rival-the-brain http://www.newyorker.com/tech/elements/a-computer-to-rival-t...

Houshalter 9y ago
In terms of energy consumed for individual computations, yes. Neurons use chemical reactions to communicate and this is terribly inefficient. Transistors use very small amounts of electricity in comparison. The main difference is computer technology is designed to be very general purpose. The brain is more like an ASIC that's hardwired to run one specific algorithm. GPUs are also computing 16 or more bits of precision, when real neurons are very low precision. There are some other differences, like how real brains are incredibly sparse and most of the synapses at any given time are dormant and not using much energy. They are also very sparsely connected to each other. While our current NNs are very dense and need to spend energy to compute every single connection each cycle.
- duncanawoods 9y ago
  It seems premature to make judgements about efficiency when there is so much we do not understand about brain function and consciousness. When you can replicate all brain function, lets compare efficiency. Comparing to an asic reveals the source of your error rather than defends your position.
Neeek 9y ago
For that to be a fair comparison, wouldn't you need to look at all the energy consumed by the human brain over the many hours it took them to become a Go champion?
- andrepd 9y ago
  Same goes for the bot, then. A back of the envelope calculation suggest Lee's brain consumed as much energy in a 80 year lifetime as AlphaGo in half a day.
  - Neeek 9y ago
    Not trying to say it isn't a correct statement, or that the outcome would be different if you lined everything up properly, only that the original statement doesn't really say anything meaningful.
- tlb 9y ago
  AlphaGo took several months of self play on a large cluster, so training probably consumed many times more energy than a human in a lifetime.
  - jacquesm 9y ago
    The gain is in being able to clone the bot perfectly. Once trained you can make many of them. Also, if you look at what happened in Chess, the lessons learned from the large machines was absorbed and resulted in your smartphone now being able to outclass the human world champion. You can expect a similar thing with Go at some point.
- yk 9y ago
  I think that's a fair argument, but from the quote above > "Last March, AlphaGo, a program created by Google DeepMind, was able to beat a world-champion human player of Go, but only after it had trained on a database of thirty million moves, running on approximately a million watts. (Its opponent’s brain, by contrast, would have been about fifty thousand times more energy-thrifty, consuming twenty watts.)" Let's say alphaGo trained for a year, that would be 1 MWyr energy consumed. And lets assume that Lee Se-dol's brain consumed 20W over 34 years of his live doing nothing but working on Go, that would be 640 Wyr, still a factor 1000-ish smaller.
charsifood 9y ago
Not surprising that a computer expends more energy to perform a task that we [previously] thought required human-like intelligence. I'm sure any dollar store calculator spends way less energy performing long division than the average human.
- intended 9y ago
  For one, normal human can do long division as fast as a calculator, and can handle numbers that will bork many calculators. (edit - look at human calculators, and the era before calculators were common place. Even now elders I know can eye ball numbers and calculate percentages / factorials and ratios) And for another, Calculation != AI, far from actually.
  - charsifood 9y ago
    One, what normal human being can perform long division as fast as a calculator? 12/43523523452. Go. Two, AI is applied statistics. What do you think AI is?
    - intended 9y ago
      You could, with practice >I'm sure any dollar store calculator spends way less energy performing long division than the average human Thats the comment. A calculator is a one role device, with exactly specified rules. Similarly, with training you can too. You don't need to be special, other than being practiced,which is a fair requirement for a human being. Here is a human being who could out perform it: https://en.wikipedia.org/wiki/Shakuntala_Devi https://en.wikipedia.org/wiki/Shakuntala_Devi >In 1977, at Southern Methodist University, she gave the 23rd root of a 201-digit number in 50 seconds.[1][4] Her answer—546,372,891—was confirmed by calculations done at the US Bureau of Standards by the UNIVAC 1101 computer, for which a special program had to be written to perform such a large calculation.[10] She could easily out-perform calculators because she never needed time to key in the commands (she needs to hear the problem to solve it). If we exclude that restriction, and the commands magically float into the calculator, and that the problem is small enough to match the calculators limits, then yes, if those arbitrary conditions are met the calculator can out-perform her brain. Which is precisely the type of “cows are round spheres” thinking that’s being decried in the article. People can and regularly do out-perform calculators in speed, energy and complexity of computation. Do note that calculators weren’t allowed as exam tools in a lot of countries till a decade or so ago. Students learnt mental math techniques which were known since ancient times (think Greece). For a human brain the answer isn’t even calculation, it becomes pattern recognition. The square root of 25 is 5, which takes about the same neural load as it takes to recognize a letter. The calculation you provided is harder, but thats a function of lack of training/practice, not complexity. ---- AI is not in the realm of what a calculator can pull off, is what I meant to say by the compute part. edit: I tried your computation on a store calculator, its beyond its ability to calculate,(0.0000000027)
madshiva 9y ago
and he only can play Go... what a waste for a such big AI. AI can beat humain on some special part but they are designed by us, then they are wrong and bad, specially when you only need to unplug the battery for that they die, too easy, come on AI do something more than that....
- rimliu 9y ago
  And can only play well on 19x19 board, if I got it right.

by wamatt 9y ago

Houshalter 9y ago

In terms of energy consumed for individual computations, yes. Neurons use chemical reactions to communicate and this is terribly inefficient. Transistors use very small amounts of electricity in comparison. The main difference is computer technology is designed to be very general purpose. The brain is more like an ASIC that's hardwired to run one specific algorithm. GPUs are also computing 16 or more bits of precision, when real neurons are very low precision. There are some other differences, like how real brains are incredibly sparse and most of the synapses at any given time are dormant and not using much energy. They are also very sparsely connected to each other. While our current NNs are very dense and need to spend energy to compute every single connection each cycle.

duncanawoods 9y ago
It seems premature to make judgements about efficiency when there is so much we do not understand about brain function and consciousness. When you can replicate all brain function, lets compare efficiency. Comparing to an asic reveals the source of your error rather than defends your position.

Neeek 9y ago

For that to be a fair comparison, wouldn't you need to look at all the energy consumed by the human brain over the many hours it took them to become a Go champion?

andrepd 9y ago
Same goes for the bot, then. A back of the envelope calculation suggest Lee's brain consumed as much energy in a 80 year lifetime as AlphaGo in half a day.
- Neeek 9y ago
  Not trying to say it isn't a correct statement, or that the outcome would be different if you lined everything up properly, only that the original statement doesn't really say anything meaningful.
tlb 9y ago
AlphaGo took several months of self play on a large cluster, so training probably consumed many times more energy than a human in a lifetime.
- jacquesm 9y ago
  The gain is in being able to clone the bot perfectly. Once trained you can make many of them. Also, if you look at what happened in Chess, the lessons learned from the large machines was absorbed and resulted in your smartphone now being able to outclass the human world champion. You can expect a similar thing with Go at some point.
yk 9y ago
I think that's a fair argument, but from the quote above > "Last March, AlphaGo, a program created by Google DeepMind, was able to beat a world-champion human player of Go, but only after it had trained on a database of thirty million moves, running on approximately a million watts. (Its opponent’s brain, by contrast, would have been about fifty thousand times more energy-thrifty, consuming twenty watts.)" Let's say alphaGo trained for a year, that would be 1 MWyr energy consumed. And lets assume that Lee Se-dol's brain consumed 20W over 34 years of his live doing nothing but working on Go, that would be 640 Wyr, still a factor 1000-ish smaller.

charsifood 9y ago

Not surprising that a computer expends more energy to perform a task that we [previously] thought required human-like intelligence. I'm sure any dollar store calculator spends way less energy performing long division than the average human.

intended 9y ago
For one, normal human can do long division as fast as a calculator, and can handle numbers that will bork many calculators. (edit - look at human calculators, and the era before calculators were common place. Even now elders I know can eye ball numbers and calculate percentages / factorials and ratios) And for another, Calculation != AI, far from actually.
- charsifood 9y ago
  One, what normal human being can perform long division as fast as a calculator? 12/43523523452. Go. Two, AI is applied statistics. What do you think AI is?
  - intended 9y ago
    You could, with practice >I'm sure any dollar store calculator spends way less energy performing long division than the average human Thats the comment. A calculator is a one role device, with exactly specified rules. Similarly, with training you can too. You don't need to be special, other than being practiced,which is a fair requirement for a human being. Here is a human being who could out perform it: https://en.wikipedia.org/wiki/Shakuntala_Devi https://en.wikipedia.org/wiki/Shakuntala_Devi >In 1977, at Southern Methodist University, she gave the 23rd root of a 201-digit number in 50 seconds.[1][4] Her answer—546,372,891—was confirmed by calculations done at the US Bureau of Standards by the UNIVAC 1101 computer, for which a special program had to be written to perform such a large calculation.[10] She could easily out-perform calculators because she never needed time to key in the commands (she needs to hear the problem to solve it). If we exclude that restriction, and the commands magically float into the calculator, and that the problem is small enough to match the calculators limits, then yes, if those arbitrary conditions are met the calculator can out-perform her brain. Which is precisely the type of “cows are round spheres” thinking that’s being decried in the article. People can and regularly do out-perform calculators in speed, energy and complexity of computation. Do note that calculators weren’t allowed as exam tools in a lot of countries till a decade or so ago. Students learnt mental math techniques which were known since ancient times (think Greece). For a human brain the answer isn’t even calculation, it becomes pattern recognition. The square root of 25 is 5, which takes about the same neural load as it takes to recognize a letter. The calculation you provided is harder, but thats a function of lack of training/practice, not complexity. ---- AI is not in the realm of what a calculator can pull off, is what I meant to say by the compute part. edit: I tried your computation on a store calculator, its beyond its ability to calculate,(0.0000000027)

madshiva 9y ago

and he only can play Go... what a waste for a such big AI. AI can beat humain on some special part but they are designed by us, then they are wrong and bad, specially when you only need to unplug the battery for that they die, too easy, come on AI do something more than that....

rimliu 9y ago
And can only play well on 19x19 board, if I got it right.