4 ms·

"even CUDA can be replaced without much disruption in most cases" Tell that to AMD who's been missing out on the very lucrative government data-center market b

"even CUDA can be replaced without much disruption in most cases" Tell that to AMD who's been missing out on the very lucrative government data-center market because they can't replace CUDA well enough. BTW, PyTorch and TensorFlow still ship without AMD support despite AMD's ROCm being around for 6 years now.

pca006132 4y ago
iirc there are comments saying that ROCm is not very mature yet. I wish OpenCL can evolve to compete with CUDA
tambourine_man 4y ago
Is that more an indictment of ROCm immaturity or of PyTorch/TensorFlow unwillingness to support it?
- fxtentacle 4y ago
  I believe ROCm isn't usable yet. I don't know anyone who successfully trained a state of the art neural network on anything that was not NVIDIA. AMD keeps offering their own forks of PyTorch and TensorFlow, but last time I checked they were years outdated, and nobody uses them. So either PyTorch/TensorFlow are secretly sponsored by NVIDIA, or AMD's and Apple's CUDA replacements aren't mature enough yet. Given how many bugs there still are in CUDA despite them being around for a long time, I'd guess that building a GPU-capable BLAS library is just a lot more work than one would think. I mean I also always wonder why CUDA comes with 400 MB of GPU shaders ... but apparently they need that to cover every edge case in a fast way. FYI, this is the ROCm "getting started" guide: https://github.com/RadeonOpenCompute/ROCm-docker/blob/master/quick-start.md https://github.com/RadeonOpenCompute/ROCm-docker/blob/master... NVIDIA has a GUI installer for CUDA.
  - pclmulqdq 4y ago
    It is the former, that Nvidia effectively sponsors ML libraries. If you're training a neural network, and you are hiring expensive ML people, you're likely just going to pay the Nvidia tax than work on ROCm support. My use of ROCm for signal processing has been pretty nice so far, and I think the ROCm BLAS library is up to par now.
  - Const-me 4y ago
    > building a GPU-capable BLAS library is just a lot more work than one would think I made a few pieces when needed: dense and sparse matrices, conjugate gradient, householder QR decomposition, other things. Wasn’t terribly hard. I only needed to support relatively new GPUs, that’s one reason why. Another one, I based on DirectCompute not ROCm. The thing’s well tested (modern videogames are using compute shaders a lot), and works with all 3 vendors of GPUs. The main downside it’s Windows only. > NVIDIA has a GUI installer for CUDA. It’s still more friction compared to D3D runtime, which is an essential OS component installed and updated as a part of the OS.
    - fxtentacle 4y ago
      DirectCompute is only the execution of fully trained neural networks, so they can skip all the gradient calculations, which are the compute-heavy and memory-heavy part. To give you and idea, I need 24GB of GPU RAM to build a background separation model, but once it's finished, it's only 500kb of parameters and it can run on CPU in JavaScript in realtime. As for the friction, CUDA is easy enough for motivated gaming kids to handle. AMD is not. But I'd predict that today's GPU enthusiast kids will become tomorrow's AI experts. As an example, look at the LAION chat which is a weird mixture of childish humor, gaming memes, and discussions about analytical approximations for probabilistic differential equations. (the Elucidated paper applied to Imagen's DDPM)
      - Const-me 4y ago
        DirectCompute is pretty close to CUDA. When handled with care, it can do these compute-heavy pieces with similar performance. The main downside is the libraries: cuBLAS, cuFFT, and similar pieces of CUDA are manually optimized, and nVidia ships multiple versions of these compute kernels for different GPUs. DirectCompute doesn’t have an equivalent, need to implement manually in HLSL. This can be tricky because for optimal performance different chips need slightly different implementations.
- pclmulqdq 4y ago
  ROCm is pretty mature, but harder to support than CUDA. Its BLAS library works well. PyTorch and TensorFlow are likely getting a lot more help from Nvidia than from AMD on compatibility. The ROCm ecosystem is a lot smaller, too, so there are many fewer examples to copy. Still, if you want to do floating point math with high performance for a lot cheaper than Nvidia, investing the time in ROCm is probably worth it.
  - fxtentacle 4y ago
    In our case, we train on NVIDIA and then port to C++ with AVX. It's a bit more work, but deployment on AMD CPUs is even cheaper than on AMD GPUs.

pca006132 4y ago

iirc there are comments saying that ROCm is not very mature yet. I wish OpenCL can evolve to compete with CUDA

tambourine_man 4y ago

Is that more an indictment of ROCm immaturity or of PyTorch/TensorFlow unwillingness to support it?

fxtentacle 4y ago
I believe ROCm isn't usable yet. I don't know anyone who successfully trained a state of the art neural network on anything that was not NVIDIA. AMD keeps offering their own forks of PyTorch and TensorFlow, but last time I checked they were years outdated, and nobody uses them. So either PyTorch/TensorFlow are secretly sponsored by NVIDIA, or AMD's and Apple's CUDA replacements aren't mature enough yet. Given how many bugs there still are in CUDA despite them being around for a long time, I'd guess that building a GPU-capable BLAS library is just a lot more work than one would think. I mean I also always wonder why CUDA comes with 400 MB of GPU shaders ... but apparently they need that to cover every edge case in a fast way. FYI, this is the ROCm "getting started" guide: https://github.com/RadeonOpenCompute/ROCm-docker/blob/master/quick-start.md https://github.com/RadeonOpenCompute/ROCm-docker/blob/master... NVIDIA has a GUI installer for CUDA.
- pclmulqdq 4y ago
  It is the former, that Nvidia effectively sponsors ML libraries. If you're training a neural network, and you are hiring expensive ML people, you're likely just going to pay the Nvidia tax than work on ROCm support. My use of ROCm for signal processing has been pretty nice so far, and I think the ROCm BLAS library is up to par now.
- Const-me 4y ago
  > building a GPU-capable BLAS library is just a lot more work than one would think I made a few pieces when needed: dense and sparse matrices, conjugate gradient, householder QR decomposition, other things. Wasn’t terribly hard. I only needed to support relatively new GPUs, that’s one reason why. Another one, I based on DirectCompute not ROCm. The thing’s well tested (modern videogames are using compute shaders a lot), and works with all 3 vendors of GPUs. The main downside it’s Windows only. > NVIDIA has a GUI installer for CUDA. It’s still more friction compared to D3D runtime, which is an essential OS component installed and updated as a part of the OS.
  - fxtentacle 4y ago
    DirectCompute is only the execution of fully trained neural networks, so they can skip all the gradient calculations, which are the compute-heavy and memory-heavy part. To give you and idea, I need 24GB of GPU RAM to build a background separation model, but once it's finished, it's only 500kb of parameters and it can run on CPU in JavaScript in realtime. As for the friction, CUDA is easy enough for motivated gaming kids to handle. AMD is not. But I'd predict that today's GPU enthusiast kids will become tomorrow's AI experts. As an example, look at the LAION chat which is a weird mixture of childish humor, gaming memes, and discussions about analytical approximations for probabilistic differential equations. (the Elucidated paper applied to Imagen's DDPM)
    - Const-me 4y ago
      DirectCompute is pretty close to CUDA. When handled with care, it can do these compute-heavy pieces with similar performance. The main downside is the libraries: cuBLAS, cuFFT, and similar pieces of CUDA are manually optimized, and nVidia ships multiple versions of these compute kernels for different GPUs. DirectCompute doesn’t have an equivalent, need to implement manually in HLSL. This can be tricky because for optimal performance different chips need slightly different implementations.
pclmulqdq 4y ago
ROCm is pretty mature, but harder to support than CUDA. Its BLAS library works well. PyTorch and TensorFlow are likely getting a lot more help from Nvidia than from AMD on compatibility. The ROCm ecosystem is a lot smaller, too, so there are many fewer examples to copy. Still, if you want to do floating point math with high performance for a lot cheaper than Nvidia, investing the time in ROCm is probably worth it.
- fxtentacle 4y ago
  In our case, we train on NVIDIA and then port to C++ with AVX. It's a bit more work, but deployment on AMD CPUs is even cheaper than on AMD GPUs.