3 ms·

> Sure, but if you've never used CUDA or any other GPU framework, how many lines of documentation do you need to read, and how many lines of code are you likely

by Kamq 3y ago

> Sure, but if you've never used CUDA or any other GPU framework, how many lines of documentation do you need to read, and how many lines of code are you likely to write and rewrite and delete before you end up with those <100 lines of working code? If you're already familiar with one of the languages that the nvidia compiler supports? Not that many. For people familiar with C or C++, it's a couple extra attributes, and a slightly different syntax for launching kernels vs a regular function call. I'm admittedly not experienced with Fortran, which is the other language they support, so I can't speak to that. There's c-style memory allocation functions, which might be annoying to C++ devs, but it's nothing that would confuse them. Edit: There's also a couple weird magic globals you have access to in a kernel (blockIdx, blockDim, threadIdx), but those are generally covered in the intros.

wiz21c 3y ago
My experience is different. I had to code various instances of "parallel reduction" and "prefix sum" and it's not easy to get into it (took me a day or two). Moreover, coming from an age where 640KB of RAM where considered quite enough, truly realizing the power of the GPU was not easy because my tasks are not quite parallel and not quite thread coherent (I grant you that doing naturally parallel stuff is dead simple). It took me a while and a lot of nvidia-nsight to max out the GPU... Moreover, I was a bit slow to actually understand how powerful a GPU is (for example, my GPU gave poor performances unless I gave it a problem big enough, so I was (wrongly) disappointed when testing toy problems) But once that challenge is overcome, GPU truly rocks. Finally debugging complex shaders (I do some specific case of computational fluid dynamics where equations are not that easy, full of "if/then" edge cases, etc) is not fun at all, tooling is sorely missed (unless I've missed something)
- Kamq 3y ago
  This is fair, and if you've got the time and inclination, I'd love to hear about your experience and the tricks you ended up pulling. There are definitely advanced areas of CUDA, and you can go deeper on nearly anything. But we are in a comment chain spawned by: > CUDA is fairly straightforward for many tasks and in many cases there is an easy 100x improvement in processing speed just sitting there to be had with <100 lines of code. And a follow up comment about how easy it would be to write that "<100 lines of code", so I feel like we're definitely talking about the easy case of naturally parallel calculations, and sticking to that as an intro seems fair to me.
  - llukas 3y ago
    There are libraries that help with more tricky stuff on-device like cub or cuFFTDx.
    - Kamq 3y ago
      Sure, but there's value in understanding how to do it yourself, even if you don't use it at work. And there's also value in seeing how other people approached a problem.
- samus 3y ago
  Parallel reduction and prefix sums are exercises to train people how to reformulate algorithms for the GPU and how to identify performance bottlenecks specific to GPUs. In practice, you'd use library functions.

by Kamq 3y ago

wiz21c 3y ago

My experience is different. I had to code various instances of "parallel reduction" and "prefix sum" and it's not easy to get into it (took me a day or two). Moreover, coming from an age where 640KB of RAM where considered quite enough, truly realizing the power of the GPU was not easy because my tasks are not quite parallel and not quite thread coherent (I grant you that doing naturally parallel stuff is dead simple). It took me a while and a lot of nvidia-nsight to max out the GPU... Moreover, I was a bit slow to actually understand how powerful a GPU is (for example, my GPU gave poor performances unless I gave it a problem big enough, so I was (wrongly) disappointed when testing toy problems) But once that challenge is overcome, GPU truly rocks. Finally debugging complex shaders (I do some specific case of computational fluid dynamics where equations are not that easy, full of "if/then" edge cases, etc) is not fun at all, tooling is sorely missed (unless I've missed something)

Kamq 3y ago
This is fair, and if you've got the time and inclination, I'd love to hear about your experience and the tricks you ended up pulling. There are definitely advanced areas of CUDA, and you can go deeper on nearly anything. But we are in a comment chain spawned by: > CUDA is fairly straightforward for many tasks and in many cases there is an easy 100x improvement in processing speed just sitting there to be had with <100 lines of code. And a follow up comment about how easy it would be to write that "<100 lines of code", so I feel like we're definitely talking about the easy case of naturally parallel calculations, and sticking to that as an intro seems fair to me.
- llukas 3y ago
  There are libraries that help with more tricky stuff on-device like cub or cuFFTDx.
  - Kamq 3y ago
    Sure, but there's value in understanding how to do it yourself, even if you don't use it at work. And there's also value in seeing how other people approached a problem.
samus 3y ago
Parallel reduction and prefix sums are exercises to train people how to reformulate algorithms for the GPU and how to identify performance bottlenecks specific to GPUs. In practice, you'd use library functions.