4 ms·

I can very well imagine there is lots of non AI/ML code that smart people could make a lot faster on a GPU. I doubt there are anywhere near enough smart people

by rocqua 4y ago

I can very well imagine there is lots of non AI/ML code that smart people could make a lot faster on a GPU. I doubt there are anywhere near enough smart people to do that. I also imagine Nvidea might like to overstate the total potential.

nobodyandproud 4y ago
It’s not that difficult. The biggest hurdles are learning the dev tech stack and setting up the development environment. The last time I tried out CUDA, it took me a couple of days to correctly setup the environment; just to spend an hour playing around parallelizing math operations. The best way I could describe the first impression dev experience: It’s a huge hack, adding non-standard language extensions to C/C++ And so relying on their own compiler or linker (iirc), instead of the standard C++. Couple this with NVidia not playing nice with OpenCL to force you into their playground, and I lost interest pretty quickly.
- rocqua 4y ago
  Besides that hack, getting your head around the way to optimize code for a GPU takes a while. Then the real challenge is taking code that is based on random access to gigabytes of RAM, and making that code work with any kind of performance on the GPU performance model. There are cases where the random access to RAM are very deep in the algorithm. Consider e.g. sparse matrix multiplication.
  - nobodyandproud 4y ago
    “optimal” is a different goalpost than “a lot faster”. Some easy-to-parallelize math and vector operations can yield impressive results without it being anywhere near optimal. I do agree that optimal requires mastery of GPU programming; but I also think GPU programming can yield good results without said mastery, and be quite accessible If anything it should be easier than writing raw multi-threading code, because GPUs aren’t (weren’t?) general purpose calculation units. So more bare metal concerns, but less to reason about overall.

by rocqua 4y ago

nobodyandproud 4y ago

It’s not that difficult. The biggest hurdles are learning the dev tech stack and setting up the development environment. The last time I tried out CUDA, it took me a couple of days to correctly setup the environment; just to spend an hour playing around parallelizing math operations. The best way I could describe the first impression dev experience: It’s a huge hack, adding non-standard language extensions to C/C++ And so relying on their own compiler or linker (iirc), instead of the standard C++. Couple this with NVidia not playing nice with OpenCL to force you into their playground, and I lost interest pretty quickly.

rocqua 4y ago
Besides that hack, getting your head around the way to optimize code for a GPU takes a while. Then the real challenge is taking code that is based on random access to gigabytes of RAM, and making that code work with any kind of performance on the GPU performance model. There are cases where the random access to RAM are very deep in the algorithm. Consider e.g. sparse matrix multiplication.
- nobodyandproud 4y ago
  “optimal” is a different goalpost than “a lot faster”. Some easy-to-parallelize math and vector operations can yield impressive results without it being anywhere near optimal. I do agree that optimal requires mastery of GPU programming; but I also think GPU programming can yield good results without said mastery, and be quite accessible If anything it should be easier than writing raw multi-threading code, because GPUs aren’t (weren’t?) general purpose calculation units. So more bare metal concerns, but less to reason about overall.