4 ms·

How do AMD gpus compare with Nvidia for deeplearning workloads? Configuration is tough enough on your own hardware with Nvidia. I have zero experience with AMD

How do AMD gpus compare with Nvidia for deeplearning workloads? Configuration is tough enough on your own hardware with Nvidia. I have zero experience with AMD cards and suspect the challenge would be greater. I do note that I saw support for AMD pretty early on last week when Stable-Diffusion was released. Nvidia may pull some pricing shenanigans for the rest of 2022 and into 2023 according to this video[1] I watched. I would be thrilled to see some real competition for workloads that are not gaming. Nvidia just dominates so far as I can tell. [1] https://www.youtube.com/watch?v=15FX4pez1dw https://www.youtube.com/watch?v=15FX4pez1dw

nolist_policy 4y ago
Very easy actually. This is not officially documented, but with a recent enough kernel you don't have to install anything. You can grab the official rocm container and it'll just work. For example for Stable Diffusion see https://github.com/AshleyYakeley/stable-diffusion-rocm/blob/master/rocm/Dockerfile https://github.com/AshleyYakeley/stable-diffusion-rocm/blob/...
deleted 4y ago
[deleted]
cercatrova 4y ago
Just don't bother with AMD if you want wide support for deep learning. While it is getting better, CUDA is king and will remain so for the future. I had an AMD GPU and I gave up and bought an Nvidia one for ML workloads. The gaming features are also superior to AMD's equivalents as well, like RTX and DLSS.
- nextaccountic 4y ago
  I just don't want to support nvidia :(
- black_puppydog 4y ago
  Out of interest, when did you try this? I just set up pytorch on a machine the other day and while I don't own an AMD GPU I saw that they offer a ROCm version on their "getting started" page [1] and it doesn't seem to be more difficult than the other options. I could imagine if you run other frameworks, or if you compile custom (c++) modules for pytorch then it could be an issue, but frankly that has been becoming exceedingly rare (at least for me). Would love to hear your or anyone's experiences since I, like the sibling comment, don't want to shove any more money to nvidia. [1]: https://pytorch.org/get-started/locally/ https://pytorch.org/get-started/locally/
  - krastanov 4y ago
    ROCm (AMD's answer to CUDA) is already pretty great on supported GPUs. For me the problem at this point is that not many of their consumer GPUs have "official" support for it (although unofficially things are a bit better).
  - homarp 4y ago
    installing ROCm is bit of a pain (there is little packaging, so you have to rebuild it yourself, and like fellow comment is saying, support is uncertain (and it does _not_ work on APU)) Search who's running Stable Diffusion on Nvidia and who's running on AMD: if you are using AMD, you are kind of on your own. Finally, you have model with custom CUDA code (e.g. https://github.com/sniklaus/3d-ken-burns https://github.com/sniklaus/3d-ken-burns )
    - mey 4y ago
      Just tried to setup Stable Diffusion tonight on Windows/AMD tonight. ROCm doesn't support Windows. Pytorch doesn't support OpenCL. WSL2 can't plug into HIP even through it's apparently hiding in Windows somewhere. Nvidia would in theory just work. Granted AMD supports SR-IOV in it's consumer GPUs, so when I have a moment, I may setup a full Hyper-V VM and pass thru the GPU. Lastly, most AI researchers simply assume you are running CUDA, and don't develop around portability.
      - terafo 4y ago
        There is DirectML build of Pytorch that should work on AMD.
      - dogma1138 4y ago
        AMD doesn’t support SR-IOV on consumer GPUs, you can do only VFIO passthrough. Open source SR-IOV support used to be accessible via GIM but there only supported GPUs upto Tonga and the closest you had for a consumer GPU was the W series of workstation cards.

nolist_policy 4y ago

Very easy actually. This is not officially documented, but with a recent enough kernel you don't have to install anything. You can grab the official rocm container and it'll just work. For example for Stable Diffusion see https://github.com/AshleyYakeley/stable-diffusion-rocm/blob/master/rocm/Dockerfile https://github.com/AshleyYakeley/stable-diffusion-rocm/blob/...

deleted 4y ago

[deleted]

cercatrova 4y ago

Just don't bother with AMD if you want wide support for deep learning. While it is getting better, CUDA is king and will remain so for the future. I had an AMD GPU and I gave up and bought an Nvidia one for ML workloads. The gaming features are also superior to AMD's equivalents as well, like RTX and DLSS.

nextaccountic 4y ago
I just don't want to support nvidia :(
black_puppydog 4y ago
Out of interest, when did you try this? I just set up pytorch on a machine the other day and while I don't own an AMD GPU I saw that they offer a ROCm version on their "getting started" page [1] and it doesn't seem to be more difficult than the other options. I could imagine if you run other frameworks, or if you compile custom (c++) modules for pytorch then it could be an issue, but frankly that has been becoming exceedingly rare (at least for me). Would love to hear your or anyone's experiences since I, like the sibling comment, don't want to shove any more money to nvidia. [1]: https://pytorch.org/get-started/locally/ https://pytorch.org/get-started/locally/
- krastanov 4y ago
  ROCm (AMD's answer to CUDA) is already pretty great on supported GPUs. For me the problem at this point is that not many of their consumer GPUs have "official" support for it (although unofficially things are a bit better).
- homarp 4y ago
  installing ROCm is bit of a pain (there is little packaging, so you have to rebuild it yourself, and like fellow comment is saying, support is uncertain (and it does _not_ work on APU)) Search who's running Stable Diffusion on Nvidia and who's running on AMD: if you are using AMD, you are kind of on your own. Finally, you have model with custom CUDA code (e.g. https://github.com/sniklaus/3d-ken-burns https://github.com/sniklaus/3d-ken-burns )
  - mey 4y ago
    Just tried to setup Stable Diffusion tonight on Windows/AMD tonight. ROCm doesn't support Windows. Pytorch doesn't support OpenCL. WSL2 can't plug into HIP even through it's apparently hiding in Windows somewhere. Nvidia would in theory just work. Granted AMD supports SR-IOV in it's consumer GPUs, so when I have a moment, I may setup a full Hyper-V VM and pass thru the GPU. Lastly, most AI researchers simply assume you are running CUDA, and don't develop around portability.
    - terafo 4y ago
      There is DirectML build of Pytorch that should work on AMD.
    - dogma1138 4y ago
      AMD doesn’t support SR-IOV on consumer GPUs, you can do only VFIO passthrough. Open source SR-IOV support used to be accessible via GIM but there only supported GPUs upto Tonga and the closest you had for a consumer GPU was the W series of workstation cards.