3 ms·

It's interesting that they focus on inference. I suppose training needs more computational power, but inference is what the end-user sees so it has harder requi

It's interesting that they focus on inference. I suppose training needs more computational power, but inference is what the end-user sees so it has harder requirements. Most of us are probably better off building a few workstations at home with high-end cards. The hardware will be more efficient for the money. But if you're considering hiring someone to manage all your machines, power-efficiency and stability become more important than the performance/upfront $ ratio. There's also FPGAs, but they tend to be much lower quality than the chips Intel or Nvidia put out so unless you know why you'd want them you don't need them.

danmaz74 10y ago
"learn" once, apply what you learned millions of times...
- dgacmu 10y ago
  Right. Or billions -- or trillions. Consider something like the Inception-like convolutional model that's one of the workloads in the paper. Training Inception is "relatively" easy -- one week of 48 K80 GPUs. (I'm lying, of course, because you retrain, and you train many times to do hyperparameter optimization, but still). Then consider the possible applications of that at Google scale -- there are "an awful lot" of images on the web, over 13PB of photos in Google photos last year [1], a gajiggle of photos in street view and google maps, an elephant worth in google plus, and probably a few trillion I'm not even thinking of. :) Same applies, of course, to Translate, and to RankBrain, also mentioned as NNs running on the TPU. 100B words per day translated [2], and .. many, many, many Google Searches per day, even if RankBrain primarily targets the 15% of never-before-seen queries [3]. Add that to the fact that GPUs are poorly-suited to realtime inference because of the large batch size requirements, and it's a solid first target. [1] https://en.wikipedia.org/wiki/Google_Photos https://en.wikipedia.org/wiki/Google_Photos [2] http://www.k-international.com/blog/google-translate-facts/ http://www.k-international.com/blog/google-translate-facts/ [3] https://www.bloomberg.com/news/articles/2015-10-26/google-turning-its-lucrative-web-search-over-to-ai-machines https://www.bloomberg.com/news/articles/2015-10-26/google-tu... (work at Google Brain on Mondays, but speakin' for myself here.)
throwaway71958 10y ago
They're also not very interested in making it easier for you to train models at home. Not that it's a big risk for them if you were able to do so: you don't have the data, and your models are only as good as your data, but they'd rather you came to their cloud and paid $2/hr per die for an outdated Tesla K80. Which, to their credit, they've made it very easy to hook up to your VM. Literally, you just tell them how many you need and your VM starts with that many GPUs attached. Super slick.
- thesandlord 10y ago
  P100s are coming soon! (I work on GCP)

danmaz74 10y ago

"learn" once, apply what you learned millions of times...

dgacmu 10y ago
Right. Or billions -- or trillions. Consider something like the Inception-like convolutional model that's one of the workloads in the paper. Training Inception is "relatively" easy -- one week of 48 K80 GPUs. (I'm lying, of course, because you retrain, and you train many times to do hyperparameter optimization, but still). Then consider the possible applications of that at Google scale -- there are "an awful lot" of images on the web, over 13PB of photos in Google photos last year [1], a gajiggle of photos in street view and google maps, an elephant worth in google plus, and probably a few trillion I'm not even thinking of. :) Same applies, of course, to Translate, and to RankBrain, also mentioned as NNs running on the TPU. 100B words per day translated [2], and .. many, many, many Google Searches per day, even if RankBrain primarily targets the 15% of never-before-seen queries [3]. Add that to the fact that GPUs are poorly-suited to realtime inference because of the large batch size requirements, and it's a solid first target. [1] https://en.wikipedia.org/wiki/Google_Photos https://en.wikipedia.org/wiki/Google_Photos [2] http://www.k-international.com/blog/google-translate-facts/ http://www.k-international.com/blog/google-translate-facts/ [3] https://www.bloomberg.com/news/articles/2015-10-26/google-turning-its-lucrative-web-search-over-to-ai-machines https://www.bloomberg.com/news/articles/2015-10-26/google-tu... (work at Google Brain on Mondays, but speakin' for myself here.)

throwaway71958 10y ago

They're also not very interested in making it easier for you to train models at home. Not that it's a big risk for them if you were able to do so: you don't have the data, and your models are only as good as your data, but they'd rather you came to their cloud and paid $2/hr per die for an outdated Tesla K80. Which, to their credit, they've made it very easy to hook up to your VM. Literally, you just tell them how many you need and your VM starts with that many GPUs attached. Super slick.

thesandlord 10y ago
P100s are coming soon! (I work on GCP)