4 ms·

I hope they improve ROCm, and add support for normal GPUs, instead of only 6800xt+. My 6700xt can't do AI, but a 3050 can, welcome to AMD drivers in the profess

I hope they improve ROCm, and add support for normal GPUs, instead of only 6800xt+. My 6700xt can't do AI, but a 3050 can, welcome to AMD drivers in the professional world.

bayindirh 4y ago
I think having loads of GPUs in the most powerful supercomputers of our era will bring in the required funding for improving it. For the last two months, huge amount of ROCm packages have been landing on Debian. I guess we'll see some improvements.
- Tuna-Fish 4y ago
  It's not a defect caused by insufficient work, but an intentional market segmentation for price discrimination. It's imho really stupid for AMD to do this given their situation in the market. At least it's fairly easy to bypass.
kkielhofner 4y ago
I’m rooting for AMD but this fundamental issue alone makes the ecosystem frustrating. CUDA will run at least run on anything that says “Nvidia” on it from the past five years or longer.
- throwaway62831 4y ago
  Not many people know that Cuda stands for Compute Unified Device Architecture. The unified being the key idea here. All NVIDIA GPUs support Cuda since the G80/G84/G86 generation which arrived at the end of 2006, beginning of 2007. It’s true of course that the older GPUs don’t support newer versions of CUDA, but the idea that CUDA is unified has been central to the project since the beginning. It also has cost a lot of money and effort for NVIDIA to put CUDA support in every GPU, even when it wasn’t extensively used. Took about 10 years of investment before it really started to pay off.
  - kkielhofner 4y ago
    All good points and yeah, my point (that I could have been clearer on) is even current versions of CUDA.
slavik81 4y ago
None of the consumer Navi 2x cards are 'officially' supported. Nevertheless, you can use the ROCm libraries anyway by setting: export HSA_OVERRIDE_GFX_VERSION=10.3.0 That will make your gfx1031 card pretend to be gfx1030, which is a supported architecture. Those processors were given different numbers in case an incompatibility was found, but I haven't heard of any thus far. Obviously, that's not as good as official support, but I hope it helps.
- Firther 4y ago
  They're not 'officially' supported like you say, but Navi21 cards (6800-6950xt) have undergone the same QA validation as the officially supported pro cards. https://github.com/RadeonOpenCompute/ROCm/issues/1714#issuecomment-1128327143 https://github.com/RadeonOpenCompute/ROCm/issues/1714#issuec...
ciupicri 4y ago
By the way, so if I have a 7900X AMD CPU I can't use ROCm with its integrated graphics?
- JonChesterfield 4y ago
  It's a rdna thing so decent chance it'll run. I'll probably know sometime this week.
spookie 4y ago
Their next GPUs will have ROCm official support btw (Navi 31 and 33). I understand your feelings though...
- jacooper 4y ago
  According to other comments, they said the same thing about the 5000 series, and they did nothing. AMD drivers on Linux are good, unless you want to do anything outside gaming.
schmorptron 4y ago
Trying to figure out the ROCM hardware support page for a solid 10 minutes and then finding out my RX 5700 which would be pretty capable hardware-wise isn't supported was super frustrating. According to some GitHub Thread GFX10 and 20 should have been supported by the end of 2021 but official support as in being listed in the document never came? I get that nvidia has a lot more resources and I'm trying not to support their closed ecosystem but AMD's non-support isn't exactly making it easy. Has anyone here had any experience with intel's new arc line?
- slavik81 4y ago
  The RX 5700 is gfx1010. You can find some of the mappings on https://llvm.org/docs/AMDGPUUsage.html https://llvm.org/docs/AMDGPUUsage.html The driver and compiler work, but the math libraries were never updated to add gfx1010, aside from rocBLAS and rocSOLVER. The official binaries don't contain machine code for your architecture, aside from those two. I would suggest building ROCm with Spack if you are using a gfx101x processor. I've been working to make sure that all of ROCm can be built for different targets. e.g. spack install --verbose --test root rocblas amdgpu_target=gfx1010 That will build rocBLAS and run a subset of the test suite. The RX 5700 hardware is not tested by ROCm QA, so running the test suite is usually a good idea. I have an RX 5700 XT available, which is also gfx1010, so if you encounter any problems and need some guidance, feel free to contact me. My email is in my profile.
  - slavik81 4y ago
    I forgot that OpenMP is broken with llvm-amdgpu in Spack at the moment. I hope it will be fixed soon, because OpenMP is used in some of the tests. In the meantime, you may have to remove `--test root` from that install command.
  - schmorptron 4y ago
    Thank you so much, I'll check this out!
dotnet00 4y ago
They sort of screwed themselves on consumer level hardware support with how they designed ROCm. Unlike CUDA, which compiles to an intermediate representation that can be compiled for the target hardware by the driver (thus allowing everything which meets the minimum feature level to run it), ROCm includes device code for each GPU, which means binary sizes would explode if they supported too many generations and every new generation needs recompiled binaries. That's also why they struggle to support new consumer hardware, why they led people on about ROCm on 5000 series until 6000 series was around the corner (this move killed my interest in taking ROCm seriously for the near future) and why they dropped support for the rx580 when it was the only consumer GPU that was still available with ROCm. They're going to have to fundamentally redesign ROCm's build process to come anywhere near CUDA's level of support.
- JonChesterfield 4y ago
  The IR per architecture is annoying. Also shipping llvm IR has hazards wrt compatibility with different llvm versions. It's solvable, probably with performance overhead.
- dagmx 4y ago
  It’s really insane to me that they target directly to hardware. Especially as a GPU vendor, they’re among the most aware of how much variances there are between different GPU designs. Pretty much every other GPU targeted language either does a runtime compilation from source or IR. This has been a known problem+solution for ages and their approach to ROCm is flummoxing.
- schmorptron 4y ago
  Wow, that's interesting. Do you think this is because they had to design this kind of system much faster than nvidia in order to release something to compete with cuda? Or are there other reasons for this sort of design decision?
  - dotnet00 4y ago
    Yeah, I think the decision was made that way because they wanted to try to catch up to CUDA but probably didn't really have good tie in with the driver team to put ROCm there. IIRC for a while you used to also have to install a custom driver to use ROCm, although at least that hasn't been necessary for some time. With ROCm on 5000 series they promised status updates several times which never came and the eventual unofficial support came after 6000 series was out. Then with the Rx580 support, they claimed that while it was unsupported it should still work and several of their developers claimed to be looking into the matter. I recall other similar incidents regarding their other smaller projects under GPUOpen. So overall it always seemed like they weren't really communicating properly internally, thus all of their projects seem somewhat disconnected from each other, leading to odd decisions like this one.
JonChesterfield 4y ago
YMMV outside of the blessed list. I did a bunch of testing with a 5700XT a while ago and it worked about as well as a card on the blessed list. If you've already tried it, how does it fail?
q-big 4y ago
> I hope they improve ROCm, and add support for normal GPUs, instead of only 6800xt+. Also, AMD should finally support ROCm under Windows. Currently, the only application known by me that uses ROCm under Windows is Blender, and they use a beta version of ROCm from AMD with Windows support for building the respective Blender releases that is not available publicly.

I hope they improve ROCm, and add support for normal GPUs, instead of only 6800xt+. My 6700xt can't do AI, but a 3050 can, welcome to AMD drivers in the professional world.

bayindirh 4y ago

I think having loads of GPUs in the most powerful supercomputers of our era will bring in the required funding for improving it. For the last two months, huge amount of ROCm packages have been landing on Debian. I guess we'll see some improvements.

Tuna-Fish 4y ago
It's not a defect caused by insufficient work, but an intentional market segmentation for price discrimination. It's imho really stupid for AMD to do this given their situation in the market. At least it's fairly easy to bypass.

kkielhofner 4y ago

I’m rooting for AMD but this fundamental issue alone makes the ecosystem frustrating. CUDA will run at least run on anything that says “Nvidia” on it from the past five years or longer.

throwaway62831 4y ago
Not many people know that Cuda stands for Compute Unified Device Architecture. The unified being the key idea here. All NVIDIA GPUs support Cuda since the G80/G84/G86 generation which arrived at the end of 2006, beginning of 2007. It’s true of course that the older GPUs don’t support newer versions of CUDA, but the idea that CUDA is unified has been central to the project since the beginning. It also has cost a lot of money and effort for NVIDIA to put CUDA support in every GPU, even when it wasn’t extensively used. Took about 10 years of investment before it really started to pay off.
- kkielhofner 4y ago
  All good points and yeah, my point (that I could have been clearer on) is even current versions of CUDA.

slavik81 4y ago

None of the consumer Navi 2x cards are 'officially' supported. Nevertheless, you can use the ROCm libraries anyway by setting: export HSA_OVERRIDE_GFX_VERSION=10.3.0 That will make your gfx1031 card pretend to be gfx1030, which is a supported architecture. Those processors were given different numbers in case an incompatibility was found, but I haven't heard of any thus far. Obviously, that's not as good as official support, but I hope it helps.

Firther 4y ago
They're not 'officially' supported like you say, but Navi21 cards (6800-6950xt) have undergone the same QA validation as the officially supported pro cards. https://github.com/RadeonOpenCompute/ROCm/issues/1714#issuecomment-1128327143 https://github.com/RadeonOpenCompute/ROCm/issues/1714#issuec...

ciupicri 4y ago

By the way, so if I have a 7900X AMD CPU I can't use ROCm with its integrated graphics?

JonChesterfield 4y ago
It's a rdna thing so decent chance it'll run. I'll probably know sometime this week.

spookie 4y ago

Their next GPUs will have ROCm official support btw (Navi 31 and 33). I understand your feelings though...

jacooper 4y ago
According to other comments, they said the same thing about the 5000 series, and they did nothing. AMD drivers on Linux are good, unless you want to do anything outside gaming.

schmorptron 4y ago

Trying to figure out the ROCM hardware support page for a solid 10 minutes and then finding out my RX 5700 which would be pretty capable hardware-wise isn't supported was super frustrating. According to some GitHub Thread GFX10 and 20 should have been supported by the end of 2021 but official support as in being listed in the document never came? I get that nvidia has a lot more resources and I'm trying not to support their closed ecosystem but AMD's non-support isn't exactly making it easy. Has anyone here had any experience with intel's new arc line?

slavik81 4y ago
The RX 5700 is gfx1010. You can find some of the mappings on https://llvm.org/docs/AMDGPUUsage.html https://llvm.org/docs/AMDGPUUsage.html The driver and compiler work, but the math libraries were never updated to add gfx1010, aside from rocBLAS and rocSOLVER. The official binaries don't contain machine code for your architecture, aside from those two. I would suggest building ROCm with Spack if you are using a gfx101x processor. I've been working to make sure that all of ROCm can be built for different targets. e.g. spack install --verbose --test root rocblas amdgpu_target=gfx1010 That will build rocBLAS and run a subset of the test suite. The RX 5700 hardware is not tested by ROCm QA, so running the test suite is usually a good idea. I have an RX 5700 XT available, which is also gfx1010, so if you encounter any problems and need some guidance, feel free to contact me. My email is in my profile.
- slavik81 4y ago
  I forgot that OpenMP is broken with llvm-amdgpu in Spack at the moment. I hope it will be fixed soon, because OpenMP is used in some of the tests. In the meantime, you may have to remove `--test root` from that install command.
- schmorptron 4y ago
  Thank you so much, I'll check this out!

dotnet00 4y ago

They sort of screwed themselves on consumer level hardware support with how they designed ROCm. Unlike CUDA, which compiles to an intermediate representation that can be compiled for the target hardware by the driver (thus allowing everything which meets the minimum feature level to run it), ROCm includes device code for each GPU, which means binary sizes would explode if they supported too many generations and every new generation needs recompiled binaries. That's also why they struggle to support new consumer hardware, why they led people on about ROCm on 5000 series until 6000 series was around the corner (this move killed my interest in taking ROCm seriously for the near future) and why they dropped support for the rx580 when it was the only consumer GPU that was still available with ROCm. They're going to have to fundamentally redesign ROCm's build process to come anywhere near CUDA's level of support.

JonChesterfield 4y ago
The IR per architecture is annoying. Also shipping llvm IR has hazards wrt compatibility with different llvm versions. It's solvable, probably with performance overhead.
dagmx 4y ago
It’s really insane to me that they target directly to hardware. Especially as a GPU vendor, they’re among the most aware of how much variances there are between different GPU designs. Pretty much every other GPU targeted language either does a runtime compilation from source or IR. This has been a known problem+solution for ages and their approach to ROCm is flummoxing.
schmorptron 4y ago
Wow, that's interesting. Do you think this is because they had to design this kind of system much faster than nvidia in order to release something to compete with cuda? Or are there other reasons for this sort of design decision?
- dotnet00 4y ago
  Yeah, I think the decision was made that way because they wanted to try to catch up to CUDA but probably didn't really have good tie in with the driver team to put ROCm there. IIRC for a while you used to also have to install a custom driver to use ROCm, although at least that hasn't been necessary for some time. With ROCm on 5000 series they promised status updates several times which never came and the eventual unofficial support came after 6000 series was out. Then with the Rx580 support, they claimed that while it was unsupported it should still work and several of their developers claimed to be looking into the matter. I recall other similar incidents regarding their other smaller projects under GPUOpen. So overall it always seemed like they weren't really communicating properly internally, thus all of their projects seem somewhat disconnected from each other, leading to odd decisions like this one.

JonChesterfield 4y ago

YMMV outside of the blessed list. I did a bunch of testing with a 5700XT a while ago and it worked about as well as a card on the blessed list. If you've already tried it, how does it fail?

q-big 4y ago

> I hope they improve ROCm, and add support for normal GPUs, instead of only 6800xt+. Also, AMD should finally support ROCm under Windows. Currently, the only application known by me that uses ROCm under Windows is Blender, and they use a beta version of ROCm from AMD with Windows support for building the respective Blender releases that is not available publicly.