4 ms·

Question from a curious sysadmin: How are these AMD GPUs for machine learning and math related stuff? I have only heard of Nvidia CUDA in this context. Does an

Question from a curious sysadmin: How are these AMD GPUs for machine learning and math related stuff? I have only heard of Nvidia CUDA in this context. Does anybody do ML work on AMD GPUs? Does the popular libraries like PyTorch, TensorFlow etc support both CUDA and AMD's equivalent?

microcolonel 8y ago
> Question from a curious sysadmin: How are these AMD GPUs for machine learning and math related stuff? They're just fine, though I think a tuned implementation on either tends to show NVIDIA ahead. > I have only heard of Nvidia CUDA in this context. Does anybody do ML work on AMD GPUs? NVIDIA is still king here, but if AMD continues to develop their Radeon Instinct stuff, it could be competitive for some applications; it then becomes a question of whether or not there will be enough people who know something other than CUDA (which is not currently supported on AMD GPUs, though work is under way to make a compatible runtime).
- celrod 8y ago
  HIP (part of the ROCm stack) is Fairlane similar to CUDA. They also have goodies like tensorflow-rocm, available through pip or a docker container. That stack is also open source. And writing in it is supposed to keep you compatible with NVidea. Wanting to support them -- and hoping they become more competitive in ML -- are why I bought Vega GPUs. Although, I haven't found much time to actually try and get any of my software running on a GPU, let alone optimized. As a huge fan of avx512, I would like to try -- maybe I can get even better performance on a graphics card. But with AMD, the lack of support for the uninitiated is apparent. Not much in the way of existing software and resources like online tutorials. I'd like guides on how to optimize kernels, organizing wavefronts and discussing memory movement. Maybe I haven't looked hard enough. Especially painful in Julia, where almost half a year after 1.0, the only supported way to use GPUs on any library is with CUDANative. "Cross platform" GPUArrays.jl's open cl backend still hasn't been updated. Means all my coding will be done in HIP. Which is fine.
sremani 8y ago
note: Not DL practitioner. As of today, if installing pytorch w/ GPU option the assumption is you have a nvidia gpu. Same thing seems to be the case for Tensorflow (did not install tensorflow lately) The first mover effect of Nvidia/CUDA should not be under-estimated.
jjoonathan 8y ago
There's a lot of CUDA-specifc code out there. Even if you think you know exactly what you need, even if it claims to be compatible with AMD cards, you should seriously consider the possibility that you are wrong on one of those counts. I was, twice. Twice I had to sell my AMD cards, buy NVIDIA cards, and eat the spread. By comparison, just paying the "green tax" up front would have been cheap. I want AMD to win, I really do, but I can't personally justify taking chances on them anymore.
jandrese 8y ago
nVidia provides way more support for the ML community than AMD, and the ML community reciprocates by buying nVidia hardware. You can use AMD stuff, but it means being on the bleeding edge and doing the debugging yourself instead of being able to ask the community for help (via Google searches). So yes you can use AMD hardware for it, but you end up making a lot more work for yourself to do so, and for little to no benefit to yourself.
jamesblonde 8y ago
ROCm is AMD's "CUDA". TensorFlow is now up-to-date - but you have to work with AMD's fork of tensorflow, they haven't upstreamed their changes yet. The new GPU of interest for DeepLearning is the MI60. It has 32GB of RAM, 1TB/s memory bandwidth. Performance on fp32 looks slightly lower than the V100 (resnet50 on tensorflow, see below). But there are no tensorcores, so resnet50 would go 3x faster using the TensorCores. Personally, i think TensorCores are not that useful in the general case, as not enough programmers that i have come across are writing their code to use them. They are good, but nichey. The important thing we don't know is the price. Expect it to be released by the end of Q1 or early Q2. Reference: https://wccftech.com/amd-radeon-mi60-resnet-benchmarks-v100-tensor-not-used/ https://wccftech.com/amd-radeon-mi60-resnet-benchmarks-v100-...

microcolonel 8y ago

> Question from a curious sysadmin: How are these AMD GPUs for machine learning and math related stuff? They're just fine, though I think a tuned implementation on either tends to show NVIDIA ahead. > I have only heard of Nvidia CUDA in this context. Does anybody do ML work on AMD GPUs? NVIDIA is still king here, but if AMD continues to develop their Radeon Instinct stuff, it could be competitive for some applications; it then becomes a question of whether or not there will be enough people who know something other than CUDA (which is not currently supported on AMD GPUs, though work is under way to make a compatible runtime).

celrod 8y ago
HIP (part of the ROCm stack) is Fairlane similar to CUDA. They also have goodies like tensorflow-rocm, available through pip or a docker container. That stack is also open source. And writing in it is supposed to keep you compatible with NVidea. Wanting to support them -- and hoping they become more competitive in ML -- are why I bought Vega GPUs. Although, I haven't found much time to actually try and get any of my software running on a GPU, let alone optimized. As a huge fan of avx512, I would like to try -- maybe I can get even better performance on a graphics card. But with AMD, the lack of support for the uninitiated is apparent. Not much in the way of existing software and resources like online tutorials. I'd like guides on how to optimize kernels, organizing wavefronts and discussing memory movement. Maybe I haven't looked hard enough. Especially painful in Julia, where almost half a year after 1.0, the only supported way to use GPUs on any library is with CUDANative. "Cross platform" GPUArrays.jl's open cl backend still hasn't been updated. Means all my coding will be done in HIP. Which is fine.

sremani 8y ago

note: Not DL practitioner. As of today, if installing pytorch w/ GPU option the assumption is you have a nvidia gpu. Same thing seems to be the case for Tensorflow (did not install tensorflow lately) The first mover effect of Nvidia/CUDA should not be under-estimated.

jjoonathan 8y ago

There's a lot of CUDA-specifc code out there. Even if you think you know exactly what you need, even if it claims to be compatible with AMD cards, you should seriously consider the possibility that you are wrong on one of those counts. I was, twice. Twice I had to sell my AMD cards, buy NVIDIA cards, and eat the spread. By comparison, just paying the "green tax" up front would have been cheap. I want AMD to win, I really do, but I can't personally justify taking chances on them anymore.

jandrese 8y ago

nVidia provides way more support for the ML community than AMD, and the ML community reciprocates by buying nVidia hardware. You can use AMD stuff, but it means being on the bleeding edge and doing the debugging yourself instead of being able to ask the community for help (via Google searches). So yes you can use AMD hardware for it, but you end up making a lot more work for yourself to do so, and for little to no benefit to yourself.

jamesblonde 8y ago

ROCm is AMD's "CUDA". TensorFlow is now up-to-date - but you have to work with AMD's fork of tensorflow, they haven't upstreamed their changes yet. The new GPU of interest for DeepLearning is the MI60. It has 32GB of RAM, 1TB/s memory bandwidth. Performance on fp32 looks slightly lower than the V100 (resnet50 on tensorflow, see below). But there are no tensorcores, so resnet50 would go 3x faster using the TensorCores. Personally, i think TensorCores are not that useful in the general case, as not enough programmers that i have come across are writing their code to use them. They are good, but nichey. The important thing we don't know is the price. Expect it to be released by the end of Q1 or early Q2. Reference: https://wccftech.com/amd-radeon-mi60-resnet-benchmarks-v100-tensor-not-used/ https://wccftech.com/amd-radeon-mi60-resnet-benchmarks-v100-...