4 ms·

Why is having a huge cluster of GPUs/CPUs so important for AlphaGo? We humans only using about 100 Watts of power. We don't need megawatts of datacenter power.

Why is having a huge cluster of GPUs/CPUs so important for AlphaGo? We humans only using about 100 Watts of power. We don't need megawatts of datacenter power. Visual SLAM is an open research problem, and lidar lets companies sidestep a lot of those problems. I think that's a reasonable bet: falling prices for an existing technology is more predictable than hoping for algorithmic/research advances.

MichaelBurge 10y ago
> Why is having a huge cluster of GPUs/CPUs so important for AlphaGo? We humans only using about 100 Watts of power. We don't need megawatts of datacenter power. This argument loses a lot of its bite after AlphaGo beats the top humans. You could argue that you still need lots of energy for training: Even a 100 watt human needs to live for 30/40 years before he's a top Go player. So it's not unreasonable that accelerating the process might use more power. And if you have all that power for training, you might as well use it for inference. You can probably build a bad Go playing machine in much less than 100 watts. There might be a way to extract something more power-efficient from AlphaGo, but it doesn't seem to be important to anyone.
- BoorishBears 10y ago
  Why can't you apply the same logic in reverse then? We're not able to speed up and compress what takes the 100 Watt machine as much as 21 years to fully develop, into something cars use locally yet, so we sidestep the problem with Lidar. Honestly both seem a little silly though, would an early, poorly trained version of Alpha GO confirm the need for lidar because it doesn't beat top humans?
- Cybiote 10y ago
  The human brain is doing the playing so we can focus on 20 watts. We can also ignore the on going visual and audio processing, complex motion planning, on line learning, signal filtering and focus on just the Go part (in other words, < 20 watts is dedicated to Go playing). Humans also are unable to learn effectively, due to limits on attention we are not sure of yet, beyond ~4 hours per day. A talented human can reach pro level in about 8 years, enough to beat a neural net only player. The total energy use in joules spent learning is still about an order of magnitude less.
- deleted 10y ago
  [deleted]
ozmodiar 10y ago
The way brains store weights, propagate updates, propagate inputs etc. is facilitated by their form and the materials they use and properties they take on. Computers like AlphaGo are essentially doing a very expensive hardware emulation of a completely different computing system. Probably some day "true" native neural network consumer hardware will exist, and I would expect it to be much more efficient. It would be a complete paradigm change from what we have now though, so it's probably going to be awhile.
- IanCal 10y ago
  A good demonstration of this is if you flip the comparison to something computers find easy. The world's fastest human seems to be able to do maybe 2 calculations per second, I've not got the details but let's assume that's a reasonable yardstick. That's 0.02 FLOPS/W. RIKEN apparently is at about 7GFLOPS/W. That's three hundred billion times more efficient. Not a fair comparison, but that's kind of the point.
- cr0sh 10y ago
  While not exactly "consumer hardware" - there do exist companies that make and sell neural network hardware; think of it as "embedded research and development" level kind of stuff (in many cases they sell development kits and the like - even to hobbyists who have the cash). For instance: http://www.cognimem.com/index.php http://www.cognimem.com/index.php http://www.general-vision.com/hardware/ http://www.general-vision.com/hardware/ ...and while not inexpensive, and not quite the same as "neural network hardware", Nvidia does sell several deep learning platforms (beyond their GPU offerings): http://www.nvidia.com/object/deep-learning-system.html http://www.nvidia.com/object/deep-learning-system.html http://www.nvidia.com/object/embedded-systems-dev-kits-modules.html http://www.nvidia.com/object/embedded-systems-dev-kits-modul... http://www.nvidia.com/object/drive-px.html http://www.nvidia.com/object/drive-px.html

MichaelBurge 10y ago

> Why is having a huge cluster of GPUs/CPUs so important for AlphaGo? We humans only using about 100 Watts of power. We don't need megawatts of datacenter power. This argument loses a lot of its bite after AlphaGo beats the top humans. You could argue that you still need lots of energy for training: Even a 100 watt human needs to live for 30/40 years before he's a top Go player. So it's not unreasonable that accelerating the process might use more power. And if you have all that power for training, you might as well use it for inference. You can probably build a bad Go playing machine in much less than 100 watts. There might be a way to extract something more power-efficient from AlphaGo, but it doesn't seem to be important to anyone.

BoorishBears 10y ago
Why can't you apply the same logic in reverse then? We're not able to speed up and compress what takes the 100 Watt machine as much as 21 years to fully develop, into something cars use locally yet, so we sidestep the problem with Lidar. Honestly both seem a little silly though, would an early, poorly trained version of Alpha GO confirm the need for lidar because it doesn't beat top humans?
Cybiote 10y ago
The human brain is doing the playing so we can focus on 20 watts. We can also ignore the on going visual and audio processing, complex motion planning, on line learning, signal filtering and focus on just the Go part (in other words, < 20 watts is dedicated to Go playing). Humans also are unable to learn effectively, due to limits on attention we are not sure of yet, beyond ~4 hours per day. A talented human can reach pro level in about 8 years, enough to beat a neural net only player. The total energy use in joules spent learning is still about an order of magnitude less.
deleted 10y ago
[deleted]

ozmodiar 10y ago

The way brains store weights, propagate updates, propagate inputs etc. is facilitated by their form and the materials they use and properties they take on. Computers like AlphaGo are essentially doing a very expensive hardware emulation of a completely different computing system. Probably some day "true" native neural network consumer hardware will exist, and I would expect it to be much more efficient. It would be a complete paradigm change from what we have now though, so it's probably going to be awhile.

IanCal 10y ago
A good demonstration of this is if you flip the comparison to something computers find easy. The world's fastest human seems to be able to do maybe 2 calculations per second, I've not got the details but let's assume that's a reasonable yardstick. That's 0.02 FLOPS/W. RIKEN apparently is at about 7GFLOPS/W. That's three hundred billion times more efficient. Not a fair comparison, but that's kind of the point.
cr0sh 10y ago
While not exactly "consumer hardware" - there do exist companies that make and sell neural network hardware; think of it as "embedded research and development" level kind of stuff (in many cases they sell development kits and the like - even to hobbyists who have the cash). For instance: http://www.cognimem.com/index.php http://www.cognimem.com/index.php http://www.general-vision.com/hardware/ http://www.general-vision.com/hardware/ ...and while not inexpensive, and not quite the same as "neural network hardware", Nvidia does sell several deep learning platforms (beyond their GPU offerings): http://www.nvidia.com/object/deep-learning-system.html http://www.nvidia.com/object/deep-learning-system.html http://www.nvidia.com/object/embedded-systems-dev-kits-modules.html http://www.nvidia.com/object/embedded-systems-dev-kits-modul... http://www.nvidia.com/object/drive-px.html http://www.nvidia.com/object/drive-px.html