6 ms·

Can somebody explain to me why they wouldn't just do this on the GPU? Isn't the GPU already designed to perform "hardware-accelerated image processing"?

bitwize 9y ago
A custom ASIC is faster still. Ask anyone in video with a RED Rocket X whether it was worth the $7000.
- dogma1138 9y ago
  RED rocket X isn’t faster than GPUs that’s been actually tested. It was mandatory for REDCINE-X Pro but since RED has implemented GPU support it’s also not really necessary.
deleted 9y ago
[deleted]
dogma1138 9y ago
On mobile GPUs likely speed and power draw custom ASICs can still be better optimized for both.
NegativeLatency 9y ago
The increased specialization of the asic probably allows some power savings compared to a more general GPU.
mtgx 9y ago
For the same reason Google's TPU was an order of magnitude more efficient than Nvidia's previous "pure" GPUs, or why video codec accelerators are also faster and more efficient than GPUs for video decoding. The GPU is still pretty "general purpose" compared to a chip that only does image processing and not much else. I hope Google will eventually reveal if the IPU shares any DNA with the TPU at all.
- make3 9y ago
  do you have an external source for this? a lot of analysts say the tpu is mostly to keep negotiation power against nvidia, that it's not really especially powerful or efficient
  - jacksmith21006 9y ago
    Gen 2 TPUs are doing 180 Tflops and a pod is doing 11.5 Peta flops so think they are pretty powerful and the real deal. https://www.nextplatform.com/2017/05/17/first-depth-look-googles-new-second-generation-tpu/ https://www.nextplatform.com/2017/05/17/first-depth-look-goo... First In-Depth Look at Google's New Second-Generation TPU
- sacheendra 9y ago
  These developments are quite funny. At one point GPUs were just fixed function hardware. As more flexibility was needed for novel applications, programmability was added. Now we are going to back to fixed function units as the calculus has changed in favour of power savings and against flexibility. I'm interesting in seeing what advance in tech will change it again in favour of flexibility.
  - ttoinou 9y ago
    Maybe because we are mixing up concepts and names ? You are comparing GPUs that were on tower computers and GPU that are now on mobile computers. Like you said, mobile require energy efficiency so having a fixed function chip makes sense right now, just like it made sense for tower computers back then.
  - photojosh 9y ago
    > the calculus has changed in favour of power savings and against flexibility This has always been a trade-off. For instance even with desktops, once video handling became common we first had video decode/encode being handled by GPU acceleration. But now most CPUs include dedicated h264 (and more recently hevc) decode/encode support, eg [0]. Although it's much lower computation, hardware acceleration is also offered for audio, and I'd take a guess the reason the Apple APIs for checking which hardware supports hardware-based audio encode/decode [1] has been deprecated is now because all supported devices provide all possible capabilities. [0]: https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video [1]: https://developer.apple.com/documentation/audiotoolbox/1620452-hardware_codec_capabilities?language=objc https://developer.apple.com/documentation/audiotoolbox/16204...
deleted 9y ago
[deleted]
cft 9y ago
That's the end f Moore's law in action. We will see more and more specialized ASICs.
TD-Linux 9y ago
I'm a little bit confused too - all the operations they describe are usually quite fast on a GPU. I wonder if it's not so much about being better at image processing, but having control and direct access to the hardware. For the Adreno, you have to go through Qualcomm's drivers and are subject to their limitations (there is freedreno which works great, but I don't think Qualcomm allows it on handsets). You're also stuck with the sizes of GPU that are available on Snapdragon SoCs - if you want a larger one, too bad.
- makomk 9y ago
  Most SoCs have some kind of dedicated image processor these days as well as a GPU, even ARM's got in on the game with their own hardware designs. Unfortunately they tend to be pretty proprietary so not much good if you want to do your own image processing. As far as I can tell, the image processors traditionally have a more DSP-like architecture (small local buffers, instruction set optimised for efficient data transfer and processing, etc), but since it's proprietary it's hard to tell. Supposedly they're meant to be more power efficient than the alternatives. (Which isn't surprising; GPUs are really designed for 3D rendering and they're definitely overkill for simpler tasks. The general assumption is that you're fetching small locally-contiguous groups of pixels from main RAM, doing texture mapping and computations on them, then conditionally blitting the result to other locally-contiguous areas of RAM. Most of the infrastructure used for this is going to waste if you're just using it to do 2D image processing.)
  - TD-Linux 9y ago
    Indeed, several good examples of these DSPs are the Qualcomm Hexagon and the VideoCore IV, the latter of which has a 64x64 register bank, and is largely reverse engineered so you can actually figure out how it works [1]. They are really good for highly serial per-block operations, such as video encoding and decoding. They would also work OK for stuff like large convolutions, which is what I kind of imagine the IPUs are for, but not really significantly better than a GPU (the low serial latency is wasted). I did find the original source to the Ars article [2] and it says that each IPU core has 512 ALUs. This seems more like a extremely wide GPU than the VPU. It also seems to be programmable in Halide [3], which presents a more "SIMT"-like interface, just like a GPU shader. It probably lacks the super fast bilinear texture mapping units that a GPU has, but otherwise seems very similar. It'd be interesting to know if the texture/pixel cache is handled automatically, or manually with a large register file like the VPU. The article also features this quote, which also seems to back Google being unsatisfied with only high level access to the GPU: >A key ingredient to the IPU’s efficiency is the tight coupling of hardware and software—our software controls many more details of the hardware than in a typical processor. It would be really awesome if they opened this chip up to third party developers. Unfortunately the press release only mentions availability in the Camera API so far, relegating it to the same opaque blob status as all the other dedicated image processors :( [1] https://github.com/hermanhermitage/videocoreiv/wiki/VideoCore-IV-Programmers-Manual https://github.com/hermanhermitage/videocoreiv/wiki/VideoCor... [2] https://blog.google/products/pixel/pixel-visual-core-image-processing-and-machine-learning-pixel-2/ https://blog.google/products/pixel/pixel-visual-core-image-p... [3] http://halide-lang.org/ http://halide-lang.org/
baybal2 9y ago
Because of NIH. Plus, I believe a separate chip allows for better lockdown

bitwize 9y ago

A custom ASIC is faster still. Ask anyone in video with a RED Rocket X whether it was worth the $7000.

dogma1138 9y ago
RED rocket X isn’t faster than GPUs that’s been actually tested. It was mandatory for REDCINE-X Pro but since RED has implemented GPU support it’s also not really necessary.

deleted 9y ago

[deleted]

dogma1138 9y ago

On mobile GPUs likely speed and power draw custom ASICs can still be better optimized for both.

NegativeLatency 9y ago

The increased specialization of the asic probably allows some power savings compared to a more general GPU.

mtgx 9y ago

For the same reason Google's TPU was an order of magnitude more efficient than Nvidia's previous "pure" GPUs, or why video codec accelerators are also faster and more efficient than GPUs for video decoding. The GPU is still pretty "general purpose" compared to a chip that only does image processing and not much else. I hope Google will eventually reveal if the IPU shares any DNA with the TPU at all.

make3 9y ago
do you have an external source for this? a lot of analysts say the tpu is mostly to keep negotiation power against nvidia, that it's not really especially powerful or efficient
- jacksmith21006 9y ago
  Gen 2 TPUs are doing 180 Tflops and a pod is doing 11.5 Peta flops so think they are pretty powerful and the real deal. https://www.nextplatform.com/2017/05/17/first-depth-look-googles-new-second-generation-tpu/ https://www.nextplatform.com/2017/05/17/first-depth-look-goo... First In-Depth Look at Google's New Second-Generation TPU
sacheendra 9y ago
These developments are quite funny. At one point GPUs were just fixed function hardware. As more flexibility was needed for novel applications, programmability was added. Now we are going to back to fixed function units as the calculus has changed in favour of power savings and against flexibility. I'm interesting in seeing what advance in tech will change it again in favour of flexibility.
- ttoinou 9y ago
  Maybe because we are mixing up concepts and names ? You are comparing GPUs that were on tower computers and GPU that are now on mobile computers. Like you said, mobile require energy efficiency so having a fixed function chip makes sense right now, just like it made sense for tower computers back then.
- photojosh 9y ago
  > the calculus has changed in favour of power savings and against flexibility This has always been a trade-off. For instance even with desktops, once video handling became common we first had video decode/encode being handled by GPU acceleration. But now most CPUs include dedicated h264 (and more recently hevc) decode/encode support, eg [0]. Although it's much lower computation, hardware acceleration is also offered for audio, and I'd take a guess the reason the Apple APIs for checking which hardware supports hardware-based audio encode/decode [1] has been deprecated is now because all supported devices provide all possible capabilities. [0]: https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video [1]: https://developer.apple.com/documentation/audiotoolbox/1620452-hardware_codec_capabilities?language=objc https://developer.apple.com/documentation/audiotoolbox/16204...

deleted 9y ago

[deleted]

cft 9y ago

That's the end f Moore's law in action. We will see more and more specialized ASICs.

TD-Linux 9y ago

I'm a little bit confused too - all the operations they describe are usually quite fast on a GPU. I wonder if it's not so much about being better at image processing, but having control and direct access to the hardware. For the Adreno, you have to go through Qualcomm's drivers and are subject to their limitations (there is freedreno which works great, but I don't think Qualcomm allows it on handsets). You're also stuck with the sizes of GPU that are available on Snapdragon SoCs - if you want a larger one, too bad.

makomk 9y ago
Most SoCs have some kind of dedicated image processor these days as well as a GPU, even ARM's got in on the game with their own hardware designs. Unfortunately they tend to be pretty proprietary so not much good if you want to do your own image processing. As far as I can tell, the image processors traditionally have a more DSP-like architecture (small local buffers, instruction set optimised for efficient data transfer and processing, etc), but since it's proprietary it's hard to tell. Supposedly they're meant to be more power efficient than the alternatives. (Which isn't surprising; GPUs are really designed for 3D rendering and they're definitely overkill for simpler tasks. The general assumption is that you're fetching small locally-contiguous groups of pixels from main RAM, doing texture mapping and computations on them, then conditionally blitting the result to other locally-contiguous areas of RAM. Most of the infrastructure used for this is going to waste if you're just using it to do 2D image processing.)
- TD-Linux 9y ago
  Indeed, several good examples of these DSPs are the Qualcomm Hexagon and the VideoCore IV, the latter of which has a 64x64 register bank, and is largely reverse engineered so you can actually figure out how it works [1]. They are really good for highly serial per-block operations, such as video encoding and decoding. They would also work OK for stuff like large convolutions, which is what I kind of imagine the IPUs are for, but not really significantly better than a GPU (the low serial latency is wasted). I did find the original source to the Ars article [2] and it says that each IPU core has 512 ALUs. This seems more like a extremely wide GPU than the VPU. It also seems to be programmable in Halide [3], which presents a more "SIMT"-like interface, just like a GPU shader. It probably lacks the super fast bilinear texture mapping units that a GPU has, but otherwise seems very similar. It'd be interesting to know if the texture/pixel cache is handled automatically, or manually with a large register file like the VPU. The article also features this quote, which also seems to back Google being unsatisfied with only high level access to the GPU: >A key ingredient to the IPU’s efficiency is the tight coupling of hardware and software—our software controls many more details of the hardware than in a typical processor. It would be really awesome if they opened this chip up to third party developers. Unfortunately the press release only mentions availability in the Camera API so far, relegating it to the same opaque blob status as all the other dedicated image processors :( [1] https://github.com/hermanhermitage/videocoreiv/wiki/VideoCore-IV-Programmers-Manual https://github.com/hermanhermitage/videocoreiv/wiki/VideoCor... [2] https://blog.google/products/pixel/pixel-visual-core-image-processing-and-machine-learning-pixel-2/ https://blog.google/products/pixel/pixel-visual-core-image-p... [3] http://halide-lang.org/ http://halide-lang.org/

baybal2 9y ago

Because of NIH. Plus, I believe a separate chip allows for better lockdown