4 ms·

For the same reason Google's TPU was an order of magnitude more efficient than Nvidia's previous "pure" GPUs, or why video codec accelerators are also faster an

by mtgx 9y ago

For the same reason Google's TPU was an order of magnitude more efficient than Nvidia's previous "pure" GPUs, or why video codec accelerators are also faster and more efficient than GPUs for video decoding. The GPU is still pretty "general purpose" compared to a chip that only does image processing and not much else. I hope Google will eventually reveal if the IPU shares any DNA with the TPU at all.

make3 9y ago
do you have an external source for this? a lot of analysts say the tpu is mostly to keep negotiation power against nvidia, that it's not really especially powerful or efficient
- jacksmith21006 9y ago
  Gen 2 TPUs are doing 180 Tflops and a pod is doing 11.5 Peta flops so think they are pretty powerful and the real deal. https://www.nextplatform.com/2017/05/17/first-depth-look-googles-new-second-generation-tpu/ https://www.nextplatform.com/2017/05/17/first-depth-look-goo... First In-Depth Look at Google's New Second-Generation TPU
sacheendra 9y ago
These developments are quite funny. At one point GPUs were just fixed function hardware. As more flexibility was needed for novel applications, programmability was added. Now we are going to back to fixed function units as the calculus has changed in favour of power savings and against flexibility. I'm interesting in seeing what advance in tech will change it again in favour of flexibility.
- ttoinou 9y ago
  Maybe because we are mixing up concepts and names ? You are comparing GPUs that were on tower computers and GPU that are now on mobile computers. Like you said, mobile require energy efficiency so having a fixed function chip makes sense right now, just like it made sense for tower computers back then.
- photojosh 9y ago
  > the calculus has changed in favour of power savings and against flexibility This has always been a trade-off. For instance even with desktops, once video handling became common we first had video decode/encode being handled by GPU acceleration. But now most CPUs include dedicated h264 (and more recently hevc) decode/encode support, eg [0]. Although it's much lower computation, hardware acceleration is also offered for audio, and I'd take a guess the reason the Apple APIs for checking which hardware supports hardware-based audio encode/decode [1] has been deprecated is now because all supported devices provide all possible capabilities. [0]: https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video [1]: https://developer.apple.com/documentation/audiotoolbox/1620452-hardware_codec_capabilities?language=objc https://developer.apple.com/documentation/audiotoolbox/16204...
  - TD-Linux 9y ago
    In the case with audio, it might be the opposite - even with a modern audio codec like Opus, an Apple CPU can decode an entire song into RAM in a fraction of a second. At that point, the CPU has to wake up anyway to download another song from Spotify or read one from disk.
dom0 9y ago
> or why video codec accelerators are also faster and more efficient than GPUs for video decoding encoders: also much higher quality
bradleyjg 9y ago
As I understand it the main advantage of the TPU is that it is built around an 8 bit pipeline. It's counter-intuitive to me, but apparently many deep learning NN can be flattened down to layers with only 256 states and still perform really well.
- gok 9y ago
  The second generation TPU is actually FP16 based.
- Symmetry 9y ago
  That's one part but doing sequencing in hardware is another big factor. And I expect they're playing tricks to reduce the read pressure on their register files.

by mtgx 9y ago

make3 9y ago

do you have an external source for this? a lot of analysts say the tpu is mostly to keep negotiation power against nvidia, that it's not really especially powerful or efficient

jacksmith21006 9y ago
Gen 2 TPUs are doing 180 Tflops and a pod is doing 11.5 Peta flops so think they are pretty powerful and the real deal. https://www.nextplatform.com/2017/05/17/first-depth-look-googles-new-second-generation-tpu/ https://www.nextplatform.com/2017/05/17/first-depth-look-goo... First In-Depth Look at Google's New Second-Generation TPU

sacheendra 9y ago

These developments are quite funny. At one point GPUs were just fixed function hardware. As more flexibility was needed for novel applications, programmability was added. Now we are going to back to fixed function units as the calculus has changed in favour of power savings and against flexibility. I'm interesting in seeing what advance in tech will change it again in favour of flexibility.

ttoinou 9y ago
Maybe because we are mixing up concepts and names ? You are comparing GPUs that were on tower computers and GPU that are now on mobile computers. Like you said, mobile require energy efficiency so having a fixed function chip makes sense right now, just like it made sense for tower computers back then.
photojosh 9y ago
> the calculus has changed in favour of power savings and against flexibility This has always been a trade-off. For instance even with desktops, once video handling became common we first had video decode/encode being handled by GPU acceleration. But now most CPUs include dedicated h264 (and more recently hevc) decode/encode support, eg [0]. Although it's much lower computation, hardware acceleration is also offered for audio, and I'd take a guess the reason the Apple APIs for checking which hardware supports hardware-based audio encode/decode [1] has been deprecated is now because all supported devices provide all possible capabilities. [0]: https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video [1]: https://developer.apple.com/documentation/audiotoolbox/1620452-hardware_codec_capabilities?language=objc https://developer.apple.com/documentation/audiotoolbox/16204...
- TD-Linux 9y ago
  In the case with audio, it might be the opposite - even with a modern audio codec like Opus, an Apple CPU can decode an entire song into RAM in a fraction of a second. At that point, the CPU has to wake up anyway to download another song from Spotify or read one from disk.

dom0 9y ago

> or why video codec accelerators are also faster and more efficient than GPUs for video decoding encoders: also much higher quality

bradleyjg 9y ago

As I understand it the main advantage of the TPU is that it is built around an 8 bit pipeline. It's counter-intuitive to me, but apparently many deep learning NN can be flattened down to layers with only 256 states and still perform really well.

gok 9y ago
The second generation TPU is actually FP16 based.
Symmetry 9y ago
That's one part but doing sequencing in hardware is another big factor. And I expect they're playing tricks to reduce the read pressure on their register files.