3 ms·

Is there a way to run things like this with an AMD graphics card? Every Stable Diffusion project I've seen seems to be CUDA focused.

habibur 4y ago
That's because Stable Diffusion is built with PyTorch. Which isn't optimized for anything but CUDA. Even the CPU is a second class citizen there. Let alone AMD or other graphics. Not saying PyTorch doesn't run on anything else. You can but those will lag and some will be hackish. Looks like Nvidia is on its way to be the next Intel.
- mrguyorama 4y ago
  Part of this is simply that AMD does a TERRIBLE job of writing tooling and software for anything that isn't just "render these triangles for this videogame". Doing raw compute things with AMD GPUs just seems limited to those building supercomputers apparently. Their promised "cross GPU" solution in ROCm is only available on a tiny fraction of the GPUs they make, seemingly without architecture excuses for why it isn't available on 5000 series cards, it took them YEARS to provide a backend that blender could actually use, productively and without crashes and bugs, and their drivers are just in general very fragile. It's weird to me how much lip service AMD puts into making cross platform, developer friendly, free and open GPU compute standards, and then turn around and just not do that.
- westurner 4y ago
  From the Arch wiki, which has a list of GPU runtimes (but not TPU or QPU runtimes) and arch package names: OpenCL, SYCL, ROCm, HIP,: https://wiki.archlinux.org/title/GPGPU https://wiki.archlinux.org/title/GPGPU : > GPGPU stands for General-purpose computing on graphics processing units. - "PyTorch OpenCL Support" https://github.com/pytorch/pytorch/issues/488 https://github.com/pytorch/pytorch/issues/488 - Blender re: removal of OpenCL support in 2021 : > The combination of the limited Cycles split kernel implementation, driver bugs, and stalled OpenCL standard has made maintenance too difficult. We can only make the kinds of bigger changes we are working on now by starting from a clean slate. We are working with AMD and Intel to get the new kernels working on their GPUs, possibly using different APIs (such as CYCL, HIP, Metal, …). - https://gitlab.com/illwieckz/i-love-compute https://gitlab.com/illwieckz/i-love-compute - https://github.com/vosen/ZLUDA https://github.com/vosen/ZLUDA - https://github.com/RadeonOpenCompute/clang-ocl https://github.com/RadeonOpenCompute/clang-ocl AMD ROCm: https://en.wikipedia.org/wiki/ROCm https://en.wikipedia.org/wiki/ROCm AMD ROcm supports Pytorch, TensorFlow, MlOpen, rocBLAS on NVIDIA and AMD GPUs: https://rocmdocs.amd.com/en/latest/Deep_learning/Deep-learning.html https://rocmdocs.amd.com/en/latest/Deep_learning/Deep-learni... RadeonOpenCompute/ROCm_Documentation: https://github.com/RadeonOpenCompute/ROCm_Documentation https://github.com/RadeonOpenCompute/ROCm_Documentation ROCm-Developer-Tools/HIPIFY https://github.com/ROCm-Developer-Tools/HIPIFY https://github.com/ROCm-Developer-Tools/HIPIFY : > hipify-clang is a clang-based tool for translating CUDA sources into HIP sources. It translates CUDA source into an abstract syntax tree, which is traversed by transformation matchers. After applying all the matchers, the output HIP source is produced. ROCmSoftwarePlatform/gpufort: https://github.com/ROCmSoftwarePlatform/gpufort https://github.com/ROCmSoftwarePlatform/gpufort : > GPUFORT: S2S translation tool for CUDA Fortran and Fortran+X in the spirit of hipify ROCm-Developer-Tools/HIP https://github.com/ROCm-Developer-Tools/HIP https://github.com/ROCm-Developer-Tools/HIP: > HIP is a C++ Runtime API and Kernel Language that allows developers to create portable applications for AMD and NVIDIA GPUs from single source code. [...] Key features include: > - HIP is very thin and has little or no performance impact over coding directly in CUDA mode. > - HIP allows coding in a single-source C++ programming language including features such as templates, C++11 lambdas, classes, namespaces, and more. > - HIP allows developers to use the "best" development environment and tools on each target platform. > - The [HIPIFY] tools automatically convert source from CUDA to HIP. > - * Developers can specialize for the platform (CUDA or AMD) to tune for performance or handle tricky cases.*
- samplatt 4y ago
  >Looks like Nvidia is on its way to be the next Intel. There's a non-trivial part of several industries who do not want to see that happen, because NVidia treats its customers and partners orders of magnitude worse than Intel ever did.
chpatrick 4y ago
You can run it on an Intel CPU if that helps: https://github.com/bes-dev/stable_diffusion.openvino https://github.com/bes-dev/stable_diffusion.openvino
sosborn 4y ago
I followed this guide and it works well: https://www.travelneil.com/stable-diffusion-windows-amd.html https://www.travelneil.com/stable-diffusion-windows-amd.html https://www.travelneil.com/stable-diffusion-updates.html https://www.travelneil.com/stable-diffusion-updates.html Tedious to set up, but the author does a great job of explaining all of the steps.
iod 4y ago
Stable Diffusion works fine for me on my RDNA2 gfx1031 ( RX6700 XT ) under Debian. If you are on a Linux kernel with amdgpu, have your distro's latest version of ROCm HIP runtime, and a relatively recent AMD GPU, you just need to replace the default pytorch with the ROCm HIP version. # Install your distro's HIP runtime and rocminfo $ apt install rocminfo hip-runtime-amd # for debian and derivatives # Confirm ROCm is installed properly and confirm your gpu is supported $ rocminfo ROCk module is loaded ... Name: gfx1031 Marketing Name: AMD Radeon RX 6700 XT ... # Must uninstall any non-ROCm torch libs $ pip3 uninstall torch torchvision torchaudio # Install latest ROCm pytorch, see website for more versions $ pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/rocm5.1.1 # Required env for unofficially supported newer gpu's like gfx1030 & gfx1031 that give error "hipErrorNoBinaryForGpu", can be added to /etc/environment to make permanent $ export HSA_OVERRIDE_GFX_VERSION=10.3.0 # Test torch HIP is working $ python3 >>> import torch >>> torch.cuda.is_available() True >>> quit() # Run Stable Diffusion workloads like normal... Hope this points people in the right direction with AMD GPUs. Blender can also take advantage of the installed HIP runtime for render acceleration in the latest versions too once you enable the setting in preferences.

habibur 4y ago

That's because Stable Diffusion is built with PyTorch. Which isn't optimized for anything but CUDA. Even the CPU is a second class citizen there. Let alone AMD or other graphics. Not saying PyTorch doesn't run on anything else. You can but those will lag and some will be hackish. Looks like Nvidia is on its way to be the next Intel.

mrguyorama 4y ago
Part of this is simply that AMD does a TERRIBLE job of writing tooling and software for anything that isn't just "render these triangles for this videogame". Doing raw compute things with AMD GPUs just seems limited to those building supercomputers apparently. Their promised "cross GPU" solution in ROCm is only available on a tiny fraction of the GPUs they make, seemingly without architecture excuses for why it isn't available on 5000 series cards, it took them YEARS to provide a backend that blender could actually use, productively and without crashes and bugs, and their drivers are just in general very fragile. It's weird to me how much lip service AMD puts into making cross platform, developer friendly, free and open GPU compute standards, and then turn around and just not do that.
westurner 4y ago
From the Arch wiki, which has a list of GPU runtimes (but not TPU or QPU runtimes) and arch package names: OpenCL, SYCL, ROCm, HIP,: https://wiki.archlinux.org/title/GPGPU https://wiki.archlinux.org/title/GPGPU : > GPGPU stands for General-purpose computing on graphics processing units. - "PyTorch OpenCL Support" https://github.com/pytorch/pytorch/issues/488 https://github.com/pytorch/pytorch/issues/488 - Blender re: removal of OpenCL support in 2021 : > The combination of the limited Cycles split kernel implementation, driver bugs, and stalled OpenCL standard has made maintenance too difficult. We can only make the kinds of bigger changes we are working on now by starting from a clean slate. We are working with AMD and Intel to get the new kernels working on their GPUs, possibly using different APIs (such as CYCL, HIP, Metal, …). - https://gitlab.com/illwieckz/i-love-compute https://gitlab.com/illwieckz/i-love-compute - https://github.com/vosen/ZLUDA https://github.com/vosen/ZLUDA - https://github.com/RadeonOpenCompute/clang-ocl https://github.com/RadeonOpenCompute/clang-ocl AMD ROCm: https://en.wikipedia.org/wiki/ROCm https://en.wikipedia.org/wiki/ROCm AMD ROcm supports Pytorch, TensorFlow, MlOpen, rocBLAS on NVIDIA and AMD GPUs: https://rocmdocs.amd.com/en/latest/Deep_learning/Deep-learning.html https://rocmdocs.amd.com/en/latest/Deep_learning/Deep-learni... RadeonOpenCompute/ROCm_Documentation: https://github.com/RadeonOpenCompute/ROCm_Documentation https://github.com/RadeonOpenCompute/ROCm_Documentation ROCm-Developer-Tools/HIPIFY https://github.com/ROCm-Developer-Tools/HIPIFY https://github.com/ROCm-Developer-Tools/HIPIFY : > hipify-clang is a clang-based tool for translating CUDA sources into HIP sources. It translates CUDA source into an abstract syntax tree, which is traversed by transformation matchers. After applying all the matchers, the output HIP source is produced. ROCmSoftwarePlatform/gpufort: https://github.com/ROCmSoftwarePlatform/gpufort https://github.com/ROCmSoftwarePlatform/gpufort : > GPUFORT: S2S translation tool for CUDA Fortran and Fortran+X in the spirit of hipify ROCm-Developer-Tools/HIP https://github.com/ROCm-Developer-Tools/HIP https://github.com/ROCm-Developer-Tools/HIP: > HIP is a C++ Runtime API and Kernel Language that allows developers to create portable applications for AMD and NVIDIA GPUs from single source code. [...] Key features include: > - HIP is very thin and has little or no performance impact over coding directly in CUDA mode. > - HIP allows coding in a single-source C++ programming language including features such as templates, C++11 lambdas, classes, namespaces, and more. > - HIP allows developers to use the "best" development environment and tools on each target platform. > - The [HIPIFY] tools automatically convert source from CUDA to HIP. > - * Developers can specialize for the platform (CUDA or AMD) to tune for performance or handle tricky cases.*
samplatt 4y ago
>Looks like Nvidia is on its way to be the next Intel. There's a non-trivial part of several industries who do not want to see that happen, because NVidia treats its customers and partners orders of magnitude worse than Intel ever did.

chpatrick 4y ago

You can run it on an Intel CPU if that helps: https://github.com/bes-dev/stable_diffusion.openvino https://github.com/bes-dev/stable_diffusion.openvino

sosborn 4y ago

I followed this guide and it works well: https://www.travelneil.com/stable-diffusion-windows-amd.html https://www.travelneil.com/stable-diffusion-windows-amd.html https://www.travelneil.com/stable-diffusion-updates.html https://www.travelneil.com/stable-diffusion-updates.html Tedious to set up, but the author does a great job of explaining all of the steps.

iod 4y ago

Stable Diffusion works fine for me on my RDNA2 gfx1031 ( RX6700 XT ) under Debian. If you are on a Linux kernel with amdgpu, have your distro's latest version of ROCm HIP runtime, and a relatively recent AMD GPU, you just need to replace the default pytorch with the ROCm HIP version. # Install your distro's HIP runtime and rocminfo $ apt install rocminfo hip-runtime-amd # for debian and derivatives # Confirm ROCm is installed properly and confirm your gpu is supported $ rocminfo ROCk module is loaded ... Name: gfx1031 Marketing Name: AMD Radeon RX 6700 XT ... # Must uninstall any non-ROCm torch libs $ pip3 uninstall torch torchvision torchaudio # Install latest ROCm pytorch, see website for more versions $ pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/rocm5.1.1 # Required env for unofficially supported newer gpu's like gfx1030 & gfx1031 that give error "hipErrorNoBinaryForGpu", can be added to /etc/environment to make permanent $ export HSA_OVERRIDE_GFX_VERSION=10.3.0 # Test torch HIP is working $ python3 >>> import torch >>> torch.cuda.is_available() True >>> quit() # Run Stable Diffusion workloads like normal... Hope this points people in the right direction with AMD GPUs. Blender can also take advantage of the installed HIP runtime for render acceleration in the latest versions too once you enable the setting in preferences.