3 ms·

> There is no moat Isn't the CUDA API a fairly significant moat?

by mort96 2y ago

roenxi 2y ago
I don't think so, no. I had an AMD card; I didn't feel like the lack of CUDA was slowing me down. I managed to implement everything I tried on the CPU just fine. My problem was that every approach I tried to implement my own stuff on the GPU using things like Vulkan or OpenCL led to crashes or frustration. In fairness I didn't know how to do basic operations on a graphics card with OpenGL and would be relying on super basic programming material out of AMD with titles like "how to multiply a matrix on an AMD GPU" [0]. They can be forgiven for not supporting amateurs I suppose ... but now I own a Nvidia card and they have stuff like https://docs.nvidia.com/cuda/cublasdx/introduction1.html https://docs.nvidia.com/cuda/cublasdx/introduction1.html. But I stress that the issue there isn't that CUDA has tutorials. It is that I expect the tutorial code not to lead to a crash or system lockup. There is a well supported path to do those basic operations. If you look at the CUDA webpage [1] you see things like "cuRAND", "cuFFT" and "cuBLAS". That isn't a moat - those are first year software engineering topics. AMD managed to make it look like a moat by not taking the compute market seriously; and anyone else competes at highly parallel compute. [0] As far as I can tell; doesn't exist. [1] https://developer.nvidia.com/cuda-zone https://developer.nvidia.com/cuda-zone
- hot_gril 2y ago
  You and a few others have said that OpenCL is buggy on AMD. Is it buggy on Nvidia too?
  - roenxi 2y ago
    I don't know. I'd use CUDA rather than experimenting with it; CUDA seems to be where all the investment is and my experience with OpenCL on AMD was so bad I don't feel the cross-platform argument would make any sense. It is a bit of a chicken-egg problem for me. I couldn't make OpenCL work on an AMD GPU, so I didn't manage to learn that much about OpenCL. At the time I assumed it was just me, but in hindsight I never saw an OpenCL-based approach to a compute problem that worked reliably on my machine so maybe it wasn't. But I don't think it really matters. The algorithms in the field don't seem to be hard and I never felt like I was struggling when implementing them on the CPU without any special API at all. My issues were conceptually similar to George Hotz's famous rants where he had crashes when running the demo app in a loop. In the experimenting phase I found I couldn't run code on the GPU with any API. I'm sure the situation improved and part of it was just me; towards the end of my time with AMD I could run stable diffusion inference and it'd work great for 10-40 minutes before the kernel paniced or whatever - so it was definitely technically possible to get a "hello world++" style thing running. But I never felt it was the APIs that were holding me back.

> There is no moat Isn't the CUDA API a fairly significant moat?

by mort96 2y ago

> There is no moat Isn't the CUDA API a fairly significant moat?

I don't think so, no. I had an AMD card; I didn't feel like the lack of CUDA was slowing me down. I managed to implement everything I tried on the CPU just fine. My problem was that every approach I tried to implement my own stuff on the GPU using things like Vulkan or OpenCL led to crashes or frustration. In fairness I didn't know how to do basic operations on a graphics card with OpenGL and would be relying on super basic programming material out of AMD with titles like "how to multiply a matrix on an AMD GPU" [0]. They can be forgiven for not supporting amateurs I suppose ... but now I own a Nvidia card and they have stuff like https://docs.nvidia.com/cuda/cublasdx/introduction1.html https://docs.nvidia.com/cuda/cublasdx/introduction1.html. But I stress that the issue there isn't that CUDA has tutorials. It is that I expect the tutorial code not to lead to a crash or system lockup. There is a well supported path to do those basic operations. If you look at the CUDA webpage [1] you see things like "cuRAND", "cuFFT" and "cuBLAS". That isn't a moat - those are first year software engineering topics. AMD managed to make it look like a moat by not taking the compute market seriously; and anyone else competes at highly parallel compute. [0] As far as I can tell; doesn't exist. [1] https://developer.nvidia.com/cuda-zone https://developer.nvidia.com/cuda-zone

hot_gril 2y ago
You and a few others have said that OpenCL is buggy on AMD. Is it buggy on Nvidia too?
- roenxi 2y ago
  I don't know. I'd use CUDA rather than experimenting with it; CUDA seems to be where all the investment is and my experience with OpenCL on AMD was so bad I don't feel the cross-platform argument would make any sense. It is a bit of a chicken-egg problem for me. I couldn't make OpenCL work on an AMD GPU, so I didn't manage to learn that much about OpenCL. At the time I assumed it was just me, but in hindsight I never saw an OpenCL-based approach to a compute problem that worked reliably on my machine so maybe it wasn't. But I don't think it really matters. The algorithms in the field don't seem to be hard and I never felt like I was struggling when implementing them on the CPU without any special API at all. My issues were conceptually similar to George Hotz's famous rants where he had crashes when running the demo app in a loop. In the experimenting phase I found I couldn't run code on the GPU with any API. I'm sure the situation improved and part of it was just me; towards the end of my time with AMD I could run stable diffusion inference and it'd work great for 10-40 minutes before the kernel paniced or whatever - so it was definitely technically possible to get a "hello world++" style thing running. But I never felt it was the APIs that were holding me back.