3 ms·
> By revealing that AlphaGo was based on this hardware Interesting, as the nature/science paper made no mention of this, it was exclusively trained on GPUs.
by arcanus 10y ago
> By revealing that AlphaGo was based on this hardware
Interesting, as the nature/science paper made no mention of this, it was exclusively trained on GPUs.
- honkhonkpants 10y agoTraining and inference are not the same thing.
- arcanus 10y agoSure... I think you are missing the point. In Nov., the Nature paper which contains the AlphaGo algorithm, the hardware detailed was exclusively CPU+GPUs. Between that time and the Lee Sedol match, the hardware running AlphaGo was switched, to these TPU. From the paper: "The final version of AlphaGo used 40 search threads, 48 CPUs, and 8 GPUs. We also implemented a distributed version of AlphaGo that exploited multiple machines, 40 search threads, 1,202 CPUs and 176 GPUs."
- dsl 10y agoOr they consider the TPU to be something similar to a math co-processor, it just offloads work from the CPU. They also didn't discuss the custom built power modules, or the custom built network switches used for interconnects.
- Merovius 10y agoNote that the version of AlphaGo that beat Fan Hui and was presented in Nature is significantly different from the Version that played Lee Sedol. Unless you believe that they didn't work on it for half a year.
- indolering 10y agoThis explains a lot. Going into the match, Sedol thought he could beat DeepMind but thought it might only be a couple of years until the technology outpaced him. We knew Fan Hui and other Go professionals were helping the team, but a massive speedup is always nice too. It's a bit underhanded, however. IMHO, the player should be able to study recent games before the match. But this is pretty typical, there were similar late-stage improvements with Chinook (checkers) and Deep Blue (chess).
- vintermann 10y agoIt's possible it was trained on GPUs but ran on TPUs.