6 ms·
Has Google talked about the amount of computing resources they're throwing at this match? I'd be very interested to know.
by agildehaus 11y ago
Has Google talked about the amount of computing resources they're throwing at this match? I'd be very interested to know.
- lololomg 11y agoThey did not say exactly but something like a couple hundred GPUs
- Eliezer 11y agoI'd guess more on the order of 10,000 GPUs.
- lololomg 11y agoThey only used 175 GPUs in the match 5 months ago.
- Eliezer 11y agoI was actually thinking primarily of distributed training time for the networks and playing time for the system, rather than the number of GPUs running this particular match. Also, I thought the number of GPUs in October was more on the order of 1,000? Happy to be told I'm mistaken though.
- Fede_V 11y agoThey used ~1000 CPUs and ~200 GPUs 5 months ago.
- deleted 11y ago[deleted]
- gearhart 11y ago1202 CPUs and 176 GPUs apparently edit: according to the livestream
- exDM69 11y ago1202 CPUs and 176 GPUs is the figure mentioned in the Nature paper. But it's important to understand that this is the computer used to train the networks used by the algorithm. It took about 30+ days worth of wallclock to train it. That's about 110 megawatt-hours (MWh) worth of energy required! During the play, the computational requirements are vastly less (but I don't know the figures). It's still probably more than is feasible to put in a smartphone in the near future. Assuming we get 3x improvement in perf per watt from going to ~20nm chips to ~7nm chips (near the theoretical minimum for silicon chips), I don't think this will work on a battery powered device. And CPUs are really bad at perf per watt on neural networks, some kind of GPU or ASIC setup will be required to make it work.
- hythloday 11y agoFor reference, it takes a little under 3 MWh to produce a car.
- tel 11y agoAre they vastly less, though? The core of the algorithm is still a deep Monte Carlo Tree Search which AlphaGo gets quite a boost on computationally for being able to fire it off in parallel. It's obviously incorrect to take the training system and assume it's identical to the live system, but I think it's disingenuous to say the live system didn't have some serious horsepower.
- PeterisP 11y agoYes, for neural networks usually training them takes many orders of magnitude more resources than just using them. For this particular example, training a system involves (1) analysis of every single game of professional go that has been digitally recorded; and (2) playing probably millions of games "against itself", both of which require far more computing power than just playing a single game.
- tel 11y agoI'm very aware of that. What I'm saying is that AlphaGo is not merely a neural net reporting best moves directly off the forward propagation. There are two nets which essentially act as proposal distributions for an exploration/exploitation tradeoff in the search space of game trees by which AlphaGo reads positions essentially out to the end of the game and ranks them by win rate (this is Monte Carlo Tree Search). The net moves are "nice" (I think they run at like 80% win rate against some other Go AIs? Maybe I'm misremembering) but the real heart of what makes AlphaGo play well is the MCTS which requires some vast resources to execute—live resources.