5 ms·

Unfortunately, despite GPUs being ubiquitous, they get very little use in the desktop/mobile space, aside from games. Apps use them as little more than blitters

Unfortunately, despite GPUs being ubiquitous, they get very little use in the desktop/mobile space, aside from games. Apps use them as little more than blitters, if they even do that much. There's a ton of computing power we're leaving on the table. Of course, improving this is a big opportunity for innovation, so perhaps it's not so unfortunate after all :)

sofaofthedamned 10y ago
Upvote for the term blitter which reminds me of my Amiga and Atari days...
- robarr 10y ago
  For the uninformed... https://en.wikipedia.org/wiki/Blitter https://en.wikipedia.org/wiki/Blitter
saganus 10y ago
I wonder if at some point (hopefully not that far from now) there's some breakthrough or development that would allow FHE in GPU. I'm guessing maybe that could be a front where the unused GPU power can be taken advantage of in desktop/mobile, besides desktop. I could imagine some sort of algorithm/driver/framework plus a protocol that would allow remote services to make queries on your personal files (without privacy issues), while being accelerated by the client's GPU. Obviously I'm talking out of my behind here but I hope maybe someday someone will figure it out.
- skdotdan 10y ago
  Excuse my ignorance, but what do you mean by FHE?
  - grzm 10y ago
    Likely Fully Homomorphic Encryption, e.g., "Accelerating fully homomorphic encryption using GPU" http://ieeexplore.ieee.org/document/6408660/ http://ieeexplore.ieee.org/document/6408660/
    - saganus 10y ago
      Aha! Very nice. This looks interesting. Even though I probably won't understand it, at least it means I wasn't that far off. Hopefully we will see more of this sooner rather than later. FHE is one of those things that I think would be a game changer if it's cheap enough to use broadly.
    - skdotdan 10y ago
      Thanks!
  - saganus 10y ago
    Yes, sorry. Fully Homomorphic Encryption https://en.wikipedia.org/wiki/Homomorphic_encryption https://en.wikipedia.org/wiki/Homomorphic_encryption
    - skdotdan 10y ago
      Thanks!
- anonymousDan 10y ago
  There has already been some academic work on accelerating fhe using GPUs, and it does indeed make a big difference. However, it's still orders of magnitude slower than native computation.
paulmd 10y ago
There's two killers, compatibility and launch overhead. CUDA is the most commonly-used compute language but it's NVIDIA-only. OpenCL is "universal" (sort of) but - being a general-purpose framework for heterogeneous compute rather than a task-specific GPGPU language - it's much more complex and lacks the libraries/etc that are available in CUDA. NVIDIA's OpenCL performance is all over the board, sometimes good sometimes fairly terrible (half of CUDA performance or less). Other users have nothing at all. So inherently - any GPU acceleration targets only a subset of your users. The other killer is launch overhead. Marshalling data into an appropriate structure, copying it over to the GPU, and copying it back takes a significant amount of time in many cases, independent of computation. This is really only viable when there is a significant amount of computation to be done, which inherently limits the number of applicable tasks. In fact ideally you would have data living right on the GPU all the time and you just make queries against it. But then you run into memory limitations - most GPUs have ~4 GB of VRAM and that's only enough VRAM for a couple applications at most (remember, we need to do a lot of work which means a fairly large quantity of data). And also since you want memory allocations to be contiguous you tend to allocate aggressively, which really means only 1-2 applications can use it. So practically speaking you now have a subset of your subset of users, the people who don't have another application running on their GPU at the same time. You can see how all this pushes it towards very niche tasks, being performed one at a time. The other approach is having an iGPU which can work in the same memory space as the CPU, like AMD does with HSA. That would lower the overhead quite a bit but right now that's a very limited subset of hardware. Practically speaking, right now the most viable general approach for vector processing is SSE/AVX, and you can see how poor the uptake of AVX has been. There's also major security concerns: GPUs aren't very conscious about things like memory segmentation or zeroing before re-allocation. As long as what you are doing is not blatantly illegal (writing to memory addresses that do not exist, etc) then it pretty much is just a single flat memory space that you can do anything with, and on non-ECC devices memory is only zeroed on a cold startup. Think "microcontroller" here, not memory like you'd get in desktop-land. https://arxiv.org/pdf/1305.7383.pdf https://arxiv.org/pdf/1305.7383.pdf https://petsymposium.org/2017/papers/issue2/paper10-2017-2-source.pdf https://petsymposium.org/2017/papers/issue2/paper10-2017-2-s... https://cryptome.org/2013/09/gpu-keylogger.pdf https://cryptome.org/2013/09/gpu-keylogger.pdf
- 10y ago

sofaofthedamned 10y ago

Upvote for the term blitter which reminds me of my Amiga and Atari days...

robarr 10y ago
For the uninformed... https://en.wikipedia.org/wiki/Blitter https://en.wikipedia.org/wiki/Blitter

saganus 10y ago

I wonder if at some point (hopefully not that far from now) there's some breakthrough or development that would allow FHE in GPU. I'm guessing maybe that could be a front where the unused GPU power can be taken advantage of in desktop/mobile, besides desktop. I could imagine some sort of algorithm/driver/framework plus a protocol that would allow remote services to make queries on your personal files (without privacy issues), while being accelerated by the client's GPU. Obviously I'm talking out of my behind here but I hope maybe someday someone will figure it out.

skdotdan 10y ago
Excuse my ignorance, but what do you mean by FHE?
- grzm 10y ago
  Likely Fully Homomorphic Encryption, e.g., "Accelerating fully homomorphic encryption using GPU" http://ieeexplore.ieee.org/document/6408660/ http://ieeexplore.ieee.org/document/6408660/
  - saganus 10y ago
    Aha! Very nice. This looks interesting. Even though I probably won't understand it, at least it means I wasn't that far off. Hopefully we will see more of this sooner rather than later. FHE is one of those things that I think would be a game changer if it's cheap enough to use broadly.
  - skdotdan 10y ago
    Thanks!
- saganus 10y ago
  Yes, sorry. Fully Homomorphic Encryption https://en.wikipedia.org/wiki/Homomorphic_encryption https://en.wikipedia.org/wiki/Homomorphic_encryption
  - skdotdan 10y ago
    Thanks!
anonymousDan 10y ago
There has already been some academic work on accelerating fhe using GPUs, and it does indeed make a big difference. However, it's still orders of magnitude slower than native computation.

paulmd 10y ago

There's two killers, compatibility and launch overhead. CUDA is the most commonly-used compute language but it's NVIDIA-only. OpenCL is "universal" (sort of) but - being a general-purpose framework for heterogeneous compute rather than a task-specific GPGPU language - it's much more complex and lacks the libraries/etc that are available in CUDA. NVIDIA's OpenCL performance is all over the board, sometimes good sometimes fairly terrible (half of CUDA performance or less). Other users have nothing at all. So inherently - any GPU acceleration targets only a subset of your users. The other killer is launch overhead. Marshalling data into an appropriate structure, copying it over to the GPU, and copying it back takes a significant amount of time in many cases, independent of computation. This is really only viable when there is a significant amount of computation to be done, which inherently limits the number of applicable tasks. In fact ideally you would have data living right on the GPU all the time and you just make queries against it. But then you run into memory limitations - most GPUs have ~4 GB of VRAM and that's only enough VRAM for a couple applications at most (remember, we need to do a lot of work which means a fairly large quantity of data). And also since you want memory allocations to be contiguous you tend to allocate aggressively, which really means only 1-2 applications can use it. So practically speaking you now have a subset of your subset of users, the people who don't have another application running on their GPU at the same time. You can see how all this pushes it towards very niche tasks, being performed one at a time. The other approach is having an iGPU which can work in the same memory space as the CPU, like AMD does with HSA. That would lower the overhead quite a bit but right now that's a very limited subset of hardware. Practically speaking, right now the most viable general approach for vector processing is SSE/AVX, and you can see how poor the uptake of AVX has been. There's also major security concerns: GPUs aren't very conscious about things like memory segmentation or zeroing before re-allocation. As long as what you are doing is not blatantly illegal (writing to memory addresses that do not exist, etc) then it pretty much is just a single flat memory space that you can do anything with, and on non-ECC devices memory is only zeroed on a cold startup. Think "microcontroller" here, not memory like you'd get in desktop-land. https://arxiv.org/pdf/1305.7383.pdf https://arxiv.org/pdf/1305.7383.pdf https://petsymposium.org/2017/papers/issue2/paper10-2017-2-source.pdf https://petsymposium.org/2017/papers/issue2/paper10-2017-2-s... https://cryptome.org/2013/09/gpu-keylogger.pdf https://cryptome.org/2013/09/gpu-keylogger.pdf

10y ago