4 ms·

I wish Google created a development version of TPU for inference so that it's possible to debug models locally and then send them for training to the GCP.

ithkuil 8y ago
An emulator?
- solomatov 8y ago
  It's too slow for this.
  - ithkuil 8y ago
    Is it? The edge TPU can do MobileNet V2 at 100 FPS. An iPhone 7 can do it at 145 FPS (source https://machinethink.net/blog/mobilenet-v2/ https://machinethink.net/blog/mobilenet-v2/) The deal is though that the Edge TPU is able to do it at much lower power.
CoolGuySteve 8y ago
Ya, I feel uneasy about this business model of creating hardware that you have to connect to a cloud service to actually use. Instead of vendor lock-in or proprietary drivers or whatever, it's a new form of locality-based lock-in. Meaning, if I have an application that needs a big hot PCI-E card attached to a physical server I own somewhere, comparable to GPUs now, the TPU is not for me. But meanwhile, a bunch of NN research and frameworks on top of TensorFlow will treat these proprietary things as a first class citizen.
- est31 8y ago
  The lock-in, while bad, only affects the development of new models. These devices exist so that you can avoid the cloud for inference.
elithrar 8y ago
The USB accelerator is designed for that - so you can target models for the a (small scale) TPU and then scale up on the cloud. This would specifically let you make sure that the TensorFlow ops your algorithms use are supported on a TPU. https://coral.withgoogle.com/products/accelerator/ https://coral.withgoogle.com/products/accelerator/
- solomatov 8y ago
  USB has too little bandwidth to do real training. Modern GPUs use 16PCIe lanes which is ~126Gpbs for PCIe 3.0 which is incomparable to USB.
  - izacus 8y ago
    Hence the part "scale up on the cloud" - USB units aren't supposed to replace the GPUs, c'mon.
  - elithrar 8y ago
    The ask here was inference, not training.
dguaraglia 8y ago
Well, you could get one of the development boards? Not sure what the use case you have in mind is, but these Edge TPUs are not for use inside of their GCP AI solutions, but rather on an 'edge' device. For anything else your 'local TPU' should be a hefty desktop with a couple of NVidia cards.
digikata 8y ago
You could buy your own, but I don't know how different the Edge TPU version is. http://linuxgizmos.com/google-launches-i-mx8m-dev-board-with-edge-tpu-ai-chip/ http://linuxgizmos.com/google-launches-i-mx8m-dev-board-with...
- solomatov 8y ago
  It's very different. It's optimized for inference not for training.
espeed 8y ago
Google Colab provides free TPU/GPU accelerated runtimes https://colab.research.google.com/ https://colab.research.google.com/
- wsxiaoys 8y ago
  TPU in Colab is not the same device as edge TPU.

ithkuil 8y ago

An emulator?

solomatov 8y ago
It's too slow for this.
- ithkuil 8y ago
  Is it? The edge TPU can do MobileNet V2 at 100 FPS. An iPhone 7 can do it at 145 FPS (source https://machinethink.net/blog/mobilenet-v2/ https://machinethink.net/blog/mobilenet-v2/) The deal is though that the Edge TPU is able to do it at much lower power.

CoolGuySteve 8y ago

Ya, I feel uneasy about this business model of creating hardware that you have to connect to a cloud service to actually use. Instead of vendor lock-in or proprietary drivers or whatever, it's a new form of locality-based lock-in. Meaning, if I have an application that needs a big hot PCI-E card attached to a physical server I own somewhere, comparable to GPUs now, the TPU is not for me. But meanwhile, a bunch of NN research and frameworks on top of TensorFlow will treat these proprietary things as a first class citizen.

est31 8y ago
The lock-in, while bad, only affects the development of new models. These devices exist so that you can avoid the cloud for inference.

elithrar 8y ago

The USB accelerator is designed for that - so you can target models for the a (small scale) TPU and then scale up on the cloud. This would specifically let you make sure that the TensorFlow ops your algorithms use are supported on a TPU. https://coral.withgoogle.com/products/accelerator/ https://coral.withgoogle.com/products/accelerator/

solomatov 8y ago
USB has too little bandwidth to do real training. Modern GPUs use 16PCIe lanes which is ~126Gpbs for PCIe 3.0 which is incomparable to USB.
- izacus 8y ago
  Hence the part "scale up on the cloud" - USB units aren't supposed to replace the GPUs, c'mon.
- elithrar 8y ago
  The ask here was inference, not training.

dguaraglia 8y ago

Well, you could get one of the development boards? Not sure what the use case you have in mind is, but these Edge TPUs are not for use inside of their GCP AI solutions, but rather on an 'edge' device. For anything else your 'local TPU' should be a hefty desktop with a couple of NVidia cards.

digikata 8y ago

You could buy your own, but I don't know how different the Edge TPU version is. http://linuxgizmos.com/google-launches-i-mx8m-dev-board-with-edge-tpu-ai-chip/ http://linuxgizmos.com/google-launches-i-mx8m-dev-board-with...

solomatov 8y ago
It's very different. It's optimized for inference not for training.

espeed 8y ago

Google Colab provides free TPU/GPU accelerated runtimes https://colab.research.google.com/ https://colab.research.google.com/

wsxiaoys 8y ago
TPU in Colab is not the same device as edge TPU.