4 ms·
Google has always been strategic about announcing what it was doing, even since the early days (I used to work there too). Think about the impact the first MapR
by mike_hearn 11y ago
Google has always been strategic about announcing what it was doing, even since the early days (I used to work there too). Think about the impact the first MapReduce and GFS/BigTable papers had.
My guess as to why they're announcing the TPU is that they are feeling the pressure from Facebook and other AI labs, and want to reinforce their reputation as being the best place to do AI research. By revealing that AlphaGo was based on this hardware, they indicate to researchers around the world that if you want to build the most advanced ML models you need to be at Google. Same reason they talked about MapReduce/GFS back in the day.
- arcanus 11y ago> By revealing that AlphaGo was based on this hardware Interesting, as the nature/science paper made no mention of this, it was exclusively trained on GPUs.
- honkhonkpants 11y agoTraining and inference are not the same thing.
- arcanus 11y agoSure... I think you are missing the point. In Nov., the Nature paper which contains the AlphaGo algorithm, the hardware detailed was exclusively CPU+GPUs. Between that time and the Lee Sedol match, the hardware running AlphaGo was switched, to these TPU. From the paper: "The final version of AlphaGo used 40 search threads, 48 CPUs, and 8 GPUs. We also implemented a distributed version of AlphaGo that exploited multiple machines, 40 search threads, 1,202 CPUs and 176 GPUs."
- dsl 11y agoOr they consider the TPU to be something similar to a math co-processor, it just offloads work from the CPU. They also didn't discuss the custom built power modules, or the custom built network switches used for interconnects.
- Merovius 11y agoNote that the version of AlphaGo that beat Fan Hui and was presented in Nature is significantly different from the Version that played Lee Sedol. Unless you believe that they didn't work on it for half a year.
- indolering 11y agoThis explains a lot. Going into the match, Sedol thought he could beat DeepMind but thought it might only be a couple of years until the technology outpaced him. We knew Fan Hui and other Go professionals were helping the team, but a massive speedup is always nice too. It's a bit underhanded, however. IMHO, the player should be able to study recent games before the match. But this is pretty typical, there were similar late-stage improvements with Chinook (checkers) and Deep Blue (chess).
- vintermann 11y agoIt's possible it was trained on GPUs but ran on TPUs.
- Krakulf 11y agoIt really boosts Google Cloud Platform with image and prestige, they're coming out with what is pretty much tensorflow as a service with their machine learning product. Coming out saying they can give you a service no one else can right down to a custom chip may sway a few buyers in the market. https://cloud.google.com/ml/ https://cloud.google.com/ml/
- hendler 11y agoFor me this is true. GPUs on Amazon are not cheap, and if google passes the savings on from performance power consumption I'd certainly offload some work to Google cloud.
- engizeer 11y agoThis is true - I've heard rumors that GCE has been trying hard to poach cloud consumers from other providers, this could be another stroke in that effort.
- manav 11y agoWell this also shows their commitment to Tensorflow if they willing to pay to fab custom chips for it. Then again, in the Google scheme of things that's probably not a huge cost. They've probably carefully modeled the energy savings over GPU/FPGA and found it to be substantial enough, even taking into account costs of design changes.
- vidarh 11y agoIf you need enough chips it doesn't take particularly careful modelling to say that you'll beat an FPGA quite substantially in most cases.
- vintermann 11y agoIt might also give Nvidia some sorely needed worries.