3 ms·

They sort of screwed themselves on consumer level hardware support with how they designed ROCm. Unlike CUDA, which compiles to an intermediate representation th

They sort of screwed themselves on consumer level hardware support with how they designed ROCm. Unlike CUDA, which compiles to an intermediate representation that can be compiled for the target hardware by the driver (thus allowing everything which meets the minimum feature level to run it), ROCm includes device code for each GPU, which means binary sizes would explode if they supported too many generations and every new generation needs recompiled binaries. That's also why they struggle to support new consumer hardware, why they led people on about ROCm on 5000 series until 6000 series was around the corner (this move killed my interest in taking ROCm seriously for the near future) and why they dropped support for the rx580 when it was the only consumer GPU that was still available with ROCm. They're going to have to fundamentally redesign ROCm's build process to come anywhere near CUDA's level of support.

JonChesterfield 4y ago
The IR per architecture is annoying. Also shipping llvm IR has hazards wrt compatibility with different llvm versions. It's solvable, probably with performance overhead.
dagmx 4y ago
It’s really insane to me that they target directly to hardware. Especially as a GPU vendor, they’re among the most aware of how much variances there are between different GPU designs. Pretty much every other GPU targeted language either does a runtime compilation from source or IR. This has been a known problem+solution for ages and their approach to ROCm is flummoxing.
schmorptron 4y ago
Wow, that's interesting. Do you think this is because they had to design this kind of system much faster than nvidia in order to release something to compete with cuda? Or are there other reasons for this sort of design decision?
- dotnet00 4y ago
  Yeah, I think the decision was made that way because they wanted to try to catch up to CUDA but probably didn't really have good tie in with the driver team to put ROCm there. IIRC for a while you used to also have to install a custom driver to use ROCm, although at least that hasn't been necessary for some time. With ROCm on 5000 series they promised status updates several times which never came and the eventual unofficial support came after 6000 series was out. Then with the Rx580 support, they claimed that while it was unsupported it should still work and several of their developers claimed to be looking into the matter. I recall other similar incidents regarding their other smaller projects under GPUOpen. So overall it always seemed like they weren't really communicating properly internally, thus all of their projects seem somewhat disconnected from each other, leading to odd decisions like this one.
  - dagmx 4y ago
    This is one of the reasons I don’t bet on AMD for GPU outside of gaming. All the other GPU vendors (NVIDIA , Intel, Apple, Qualcomm etc ) are investing strategically on making sure popular software is hardware accelerated by their products. NVIDIA is clearly in the lead here, due to the excellent choices for CUDA, but AMD is the only vendor who seems to not be pushing forward here because their HIP and ROCm strategy seems to be flawed.
    - ColonelPhantom 4y ago
      On the other hand, I don't think any other vendors aim for compatibility with CUDA. Intel is laser focused on oneAPI which is just SYCL iirc. Sure, SYCL is cool and all, but you cannot trivially translate most CUDA programs to SYCL like you can with HIP.
dogma1138 4y ago
It’s worse because there is no IR in the driver there is zero guarantee for forward compatibility and you need create binaries for every hardware for backwards compatibility. So not only does it mean that you have to choose which hardware you want to support at any point in time but you have to maintain your codebase and release new binaries every time AMD releases a new GPU. And it gets even more complicated because even intra-generation compatibility isn’t granted since differ GPUs from the same generation can have slight variances in them that essentially requires you to target them specifically. On the other hand CUDA binaries that date back to the days of Tesla and Fermi can still run on current hardware with no issues. The architecture behind ROCm does not make any sense outside of custom implementations for supercomputers and bespoke hyperscaler size deployments.

JonChesterfield 4y ago

The IR per architecture is annoying. Also shipping llvm IR has hazards wrt compatibility with different llvm versions. It's solvable, probably with performance overhead.

dagmx 4y ago

It’s really insane to me that they target directly to hardware. Especially as a GPU vendor, they’re among the most aware of how much variances there are between different GPU designs. Pretty much every other GPU targeted language either does a runtime compilation from source or IR. This has been a known problem+solution for ages and their approach to ROCm is flummoxing.

schmorptron 4y ago

Wow, that's interesting. Do you think this is because they had to design this kind of system much faster than nvidia in order to release something to compete with cuda? Or are there other reasons for this sort of design decision?

dotnet00 4y ago
Yeah, I think the decision was made that way because they wanted to try to catch up to CUDA but probably didn't really have good tie in with the driver team to put ROCm there. IIRC for a while you used to also have to install a custom driver to use ROCm, although at least that hasn't been necessary for some time. With ROCm on 5000 series they promised status updates several times which never came and the eventual unofficial support came after 6000 series was out. Then with the Rx580 support, they claimed that while it was unsupported it should still work and several of their developers claimed to be looking into the matter. I recall other similar incidents regarding their other smaller projects under GPUOpen. So overall it always seemed like they weren't really communicating properly internally, thus all of their projects seem somewhat disconnected from each other, leading to odd decisions like this one.
- dagmx 4y ago
  This is one of the reasons I don’t bet on AMD for GPU outside of gaming. All the other GPU vendors (NVIDIA , Intel, Apple, Qualcomm etc ) are investing strategically on making sure popular software is hardware accelerated by their products. NVIDIA is clearly in the lead here, due to the excellent choices for CUDA, but AMD is the only vendor who seems to not be pushing forward here because their HIP and ROCm strategy seems to be flawed.
  - ColonelPhantom 4y ago
    On the other hand, I don't think any other vendors aim for compatibility with CUDA. Intel is laser focused on oneAPI which is just SYCL iirc. Sure, SYCL is cool and all, but you cannot trivially translate most CUDA programs to SYCL like you can with HIP.

dogma1138 4y ago

It’s worse because there is no IR in the driver there is zero guarantee for forward compatibility and you need create binaries for every hardware for backwards compatibility. So not only does it mean that you have to choose which hardware you want to support at any point in time but you have to maintain your codebase and release new binaries every time AMD releases a new GPU. And it gets even more complicated because even intra-generation compatibility isn’t granted since differ GPUs from the same generation can have slight variances in them that essentially requires you to target them specifically. On the other hand CUDA binaries that date back to the days of Tesla and Fermi can still run on current hardware with no issues. The architecture behind ROCm does not make any sense outside of custom implementations for supercomputers and bespoke hyperscaler size deployments.