9 ms·
But not, apparently, ahead of what you can do with 8 consumer GPUs
by habitue 10y ago
But not, apparently, ahead of what you can do with 8 consumer GPUs
- Houshalter 10y ago>On WMT En→Fr, it takes around 6 days to train a basic model using 96 NVIDIA K80 GPUs. Then they use custom ASICs to actually run the thing in production.
- aab0 10y agoA $3k Tesla K80 GPU is not 'consumer', nor do most consumers (or most researchers or small businesses) have $24k to drop on a set of GPUs alone.
- habitue 10y agoAh, sorry I missed that they were tesla gpus
- dgacmu 10y agoYou're right - a $3k k80 is generally inferior for most DNN applications than a $1k Titan X. The primary reasons that big companies use K80s have to do with achieving high computational density, and licensing issues, more than the performance of a single GPU. Sticking 8 Titan X boards in a machine is a bummer job if you want to pack them closely together. But for academic researchers, a quad Titan X box is pretty solid and quite affordable.
- aab0 10y agoA quad Titan X is still $4k for the GPUs alone, and was only possible in the past few months - people might have wanted to get stuff done in the years in between the last generation and the current generation...
- ma2rten 10y ago"On WMT En → Fr, it takes around 6 days to train a basic model using 96 NVIDIA K80 GPUs" Notice that the actual Google Translate uses a much bigger dataset than that.
- visarga 10y agoThat would make the cost about $15..30K for a model.
- visarga 10y ago> The training setup follows the classic data parallelism paradigm. There are 12 replicas running concurrently on separate machines. Every replica updates the shared parameters asynchronously. So it's actually 96 GPUs for a model. And they have many languages to train, and probably don't use cheap GPUs, either.
- TulliusCicero 10y agoIf it's just expensive to train the model, I wouldn't expect that to stop Google. They don't have to do the actual training that often, after all. Right?
- visarga 10y agoThey would have to do that for each language (or language pair), but they can afford it. I think the training the network is cheap compared to running it for user queries, at Google scale.