3 ms·

This page has the first mention of Google TPUs since the initial announcement from Google. Anyone know what the status is? When TPUs will be allowed to be used

by ktta 10y ago

This page has the first mention of Google TPUs since the initial announcement from Google. Anyone know what the status is? When TPUs will be allowed to be used in Google Cloud? I'm confused as to why they ever announced it at a big event like Google I/O rather than a paper or even a simple blog post if they aren't going to give people access to them. There's some hint of it being offered in conjunction with TF and other ML cloud offerings in the blog post[1], and this 'XLA compiler framework' looks it's related. But I'm still wondering how much time people have to wait. [1]:https://cloudplatform.googleblog.com/2016/05/Google-supercharges-machine-learning-tasks-with-custom-chip.html https://cloudplatform.googleblog.com/2016/05/Google-supercha...

visarga 10y ago
> When TPUs will be allowed to be used in Google Cloud? TPUs are only useful for prediction, not training, and for very high volumes of work where energy use is a major cost, so, it's for Google and FaceBook-type of situations. They are not designed as accelerated general purpose processors for deep learning. Cheap inference can be done on CPUs as well, because production models can be optimized to only a small fraction of the computation (20x reductions are not impossible) by pruning neurons, weight quantization and other techniques.
- dharma1 10y ago
  CPU/mobile inference - I think we will see a lot more stuff like this this year - https://xnor.ai/ https://xnor.ai/
- halflings 10y ago
  What leads you to think that TPUs are not useful for training?
  - alfalfasprout 10y ago
    Probably memory. Unless their ASICs have some kind of ultra fast on-chip memory. That's why GPUs still reign supreme. FPGAs have started seeing use for inference for those of us without the billion dollar budgets needed to develop ASICs.
  - ogrisel 10y ago
    The aforementioned google translate paper suggests so. My hunch is that TPUs work with a mix of 8 bit and 16 bit integer arithmetic on quantized networks with quantized data [1]. As far as I know nobody has managed to make SGD work properly with integer weights and that could be a reason why TPUs are not used for training (yet). [1] https://petewarden.com/2016/05/03/how-to-quantize-neural-networks-with-tensorflow/ https://petewarden.com/2016/05/03/how-to-quantize-neural-net... I could be completely wrong though.
ma2rten 10y ago
Google Translate Paper mentions TPUs as well: https://arxiv.org/abs/1609.08144 https://arxiv.org/abs/1609.08144

by ktta 10y ago

visarga 10y ago

> When TPUs will be allowed to be used in Google Cloud? TPUs are only useful for prediction, not training, and for very high volumes of work where energy use is a major cost, so, it's for Google and FaceBook-type of situations. They are not designed as accelerated general purpose processors for deep learning. Cheap inference can be done on CPUs as well, because production models can be optimized to only a small fraction of the computation (20x reductions are not impossible) by pruning neurons, weight quantization and other techniques.

dharma1 10y ago
CPU/mobile inference - I think we will see a lot more stuff like this this year - https://xnor.ai/ https://xnor.ai/
halflings 10y ago
What leads you to think that TPUs are not useful for training?
- alfalfasprout 10y ago
  Probably memory. Unless their ASICs have some kind of ultra fast on-chip memory. That's why GPUs still reign supreme. FPGAs have started seeing use for inference for those of us without the billion dollar budgets needed to develop ASICs.
- ogrisel 10y ago
  The aforementioned google translate paper suggests so. My hunch is that TPUs work with a mix of 8 bit and 16 bit integer arithmetic on quantized networks with quantized data [1]. As far as I know nobody has managed to make SGD work properly with integer weights and that could be a reason why TPUs are not used for training (yet). [1] https://petewarden.com/2016/05/03/how-to-quantize-neural-networks-with-tensorflow/ https://petewarden.com/2016/05/03/how-to-quantize-neural-net... I could be completely wrong though.

ma2rten 10y ago

Google Translate Paper mentions TPUs as well: https://arxiv.org/abs/1609.08144 https://arxiv.org/abs/1609.08144