5 ms·

I wish AMD would just drop ROCm at this stage, and focus on SYCL. The rocRAND/hipRAND woes in this article are if anything showing ROCm in a better light than i

I wish AMD would just drop ROCm at this stage, and focus on SYCL. The rocRAND/hipRAND woes in this article are if anything showing ROCm in a better light than it really is; here it at least worked and performed within the same ballpark as CUDA. Often it simply does not work at all, or if it works it's behind by a lot more. At work I simply gave up on our 4x Radeon Pro W6800 workstation because launching Tensorflow with more than 1 GPU would cause a kernel panic every time, and AMD engineers never offered a fix other than "reinstall Ubuntu". ROCm feels like such a half assed product that (to me at least) feels like it's been made to tick a box and look cool in corporate presentations. It's not made with the proper mindset to compete against CUDA. Lisa Su claims they're doubling down on ROCm but to me it feels like they're falling behind relative to Nvidia, not catching up. Banding together with Intel to support SYCL would in my opinion 1. Ensure there's a lot more momentum behind a single, cross-platform, industry-standard competitor 2. Entice other industry heavyweights like MSFT, Qualcomm, ARM etc to also take the cross-platform solutions more seriously 3. Encourage heavy investment into the developer experience and tooling for the cross-platform solution

arcanus 3y ago
SYCL is a terrible option. The performance of any application on SYCL is currently quite poor. Why drop ROCm (used on the world's largest supercomputer?) for an even more unproven product that AMD has no control over the roadmap? > Banding together with Intel to support SYCL would in my opinion Except that Intel has more control over SYCL and has repeatedly hurt AMD products with anticompetitive behavior in the past. Why would AMD permit their software to be controlled be a competitor? AMD is executing with ROCm NOW, with multiple Top500 supercomputer wins and deployment in major cloud vendors with MI300X. Yes, the software needs to improve, but the practice of throwing out software to start over is not a good strategy. AMD has far more momentum in the data center than INTC at present.
- shihab 3y ago
  I agree about ROCm. When it comes to comparison to SYCL, HIP is much closer to the spirit of SYCL than ROCm is. Both aim to help writing a single codebase that'll run across multiple hardwares. For now though, the trajectory of SYCL appears much more promising to me than HIP. HIP is already split in two parts for CPU and GPU, which is baffling, and neither part seems to receive much love from AMD.
- pjmlp 3y ago
  It hardly matters if most desktops and laptops will be running CUDA/SYCL instead.
- hjabird 3y ago
  > The performance of any application on SYCL is currently quite poor. SYCL can get pretty much equivalent performance in Kernels to eg. CUDA. Try looking at SYCL performance papers on Arxiv. Eg. see [1]. That isn't to say that SYCL code is optimised on every platform without tweaking - you do still need to put effort into target specific optimizations to get the best performance, like you would in the CUDA or HIP. > Why drop ROCm (used on the world's largest supercomputer?) Some of the world's largest super-computers / HPC applications do use SYCL for AMD! The application I'm most aware of for this is GROMACS. As to why? - because having 3 version of the same code using different programming APIs is a big maintenance burden. [1] https://arxiv.org/pdf/2309.09609.pdf https://arxiv.org/pdf/2309.09609.pdf
  - arcanus 3y ago
    > Some of the world's largest super-computers / HPC applications do use SYCL for AMD! The application I'm most aware of for this is GROMACS. As to why? - because having 3 version of the same code using different programming APIs is a big maintenance burden. The fact that GROMACS is unwilling to drop CUDA support to stand fully behind SYCL is very telling.
    - hjabird 3y ago
      I wouldn't expect them to drop CUDA support, even if SYCL is a viable alternative: * The CUDA backend is mature, featureful, and significant effort has been invested into optimising it on Nvidia hardware. One does not simply throw away a performant, well validated HPC code! * Nvidia GPUs dominate the GPGPU market - unlike AMD's. * The SYCL backend is still is very new in comparison (they even state in the docs to pay extra attention to validation), and doesn't have Nvidia-specific optimisations yet. Why prioritise reimplementing what already exists?
cyanydeez 3y ago
hate to break it to you but with the mi300, they're clearly going in the same direction. I looked at their current compatibility chart for rocm and it's clearly just starting.
pokeypokes 3y ago
Important to understand that with these kinds of libraries, it's one thing to get everything working, and a whole other to have it all be fast. Given finite resources AMD are (rightfully) focused on the second, for a narrower set of use cases - namely big customers with big pockets, using the latest compute cards. See, for example, the world's fastest super computer. That's a whole lot more than a presentation tick box. As someone doing relatively small scale work, on gaming cards, you just aren't the target user. While these kinds of users are well represented on forums, they're a rounding error in terms of actual $. Switching to SYCL makes no sense. They need to unseat Nvidia, and specifically CUDA. The whole point of HIP is to clone CUDA, make it easy for users to switch and piggy back on Nvidias success. I used to work in this space (not at AMD lol), and generally I don't think the comments on HN are fair to AMD. Obviously ROCm has a long way to go (CUDA and libs are some truly amazing work), but it's going.
- throwaway81523 3y ago
  Don't know about the libs but CUDA itself is a C++ dialect targeted at a family of quirky array processors (GPU's). That takes some dev work by compiler hackers but doesn't seem amazing. Am I missing something? It's of course still easy to mess up such a project by not bringing enough clue, and AMD might have done that with ROCm. Is that what happened, or is it something different?
  - pjmlp 3y ago
    CUDA is more than a C++ dialect, which is a big thing people keep missing with all those "CUDA replacements". Until CUDA 3.0, it was similar to OpenCL, a C dialect, however afterwards it became a C, C++ dialect, with common infrastructure PTX. PGI targeted PTX, with their C, C++, and very relevant, Fortran compilers for HPC. PGI was acquired by NVidia, and became the main set of CUDA compilers. Given PTX, many other languages started targeting CUDA as well, Java, .NET, Haskell, Julia, at very least. NVidia is now invested into a Python JIT for CUDA as well. So yeah, while C++20 is the main language in CUDA, there is also a whole ecosystem of programming languages, that the "CUDA replacements" keep ignoring.
kimixa 3y ago
I feel this is falling into the rewrite trap - rather than continuing to develop an existing but imperfect system, throw it all away for something that is unproven and currently in a notably worse state. What about rocm makes it fundamentally by design that much worse? And what about sycl makes it better? Rocm is already based on many standard shared OSS projects, like llvm and clang. Often the same ones SYCL stacks are based on. What about SYCL makes it so superior that it'll make up for throwing all the current work away?
- imtringued 3y ago
  Most likely the ability to run software on the GPU. When I last used ROCm I got both bad performance and it was prone to crashing.
  - JonChesterfield 3y ago
    It's really sensitive to the (Linux) driver/compiler/library versions all lining up. If you get the blessed Ubuntu kernel version and use it with the exact corresponding driver from dkms and the matching userspace version it's reasonably solid. If you use the driver in Linux upstream, whether you have a good time or a terrible one varies with the age of the userspace libraries you're pairing with it. I don't know what to do about this setup. I think it's an artifact of gearing testing towards "releases" of the entire stack with specific HPC operating systems in mind. To do this right, each component would need to be individually solid and deal with version/interface slip relative to other components, which means the cross product of testing patterns is rapidly infeasible. I'm thinking about it but don't see a solution yet.
- cherryteastain 3y ago
  > What about rocm makes it fundamentally by design that much worse? And what about sycl makes it better? The fact that SYCL compiles device code to SPIR-V instead of a device specific ISA like HIP for instance. The decision to do the latter, alone, is what causes ROCm to have support for a very narrow range of cards which significantly hampers adoption.
zozbot234 3y ago
SYCL is practically comparable to OpenCL which is already getting unofficial support (including on older hardware that Rocm does not support) via Mesa's RustiCL. Additionally clspv and Mesa's Zink are both exploring OpenCL (and potentially SYCL)-on-Vulkan, though that option involves lots of quirks since the standards have evolved independently (and are even based on different varieties of SPIR-V).
JonChesterfield 3y ago
A few issues with that. One is that ROCm is the entire compute stack - driver, libraries, compilers, languages and sycl is a programming language. Sycl is more like one more language that could be built on ROCm than a replacement for it. The second is that all the entire target market wrote all their stuff in cuda and looks totally unwilling to change to anything else. That makes HIP look like a good idea and Sycl not so much. Noone (commercially significant) is likely to port their code to Sycl to benefit from Intel's GPUs. It's worth saying that AMD bet heavily on cross platform technology. Initially opencl, but take a look at the companies behind HSA as well. I'd say that doesn't appear to have worked at all - they've got the hassle of working with external specifications, and other companies have given up on the specs, so really noone is winning there. I think Triton is a cross company thing currently being implemented. Finally Intel specifically is moribund. Definitely don't want to be relying on them to do anything. On the bright side, the wide array of GPU programming languages are all very similar to each other from the perspective of the compiler stack, e.g. they all run through the same LLVM back end. So if someone decides they like sycl on amdgpu enough to write it, a lot of existing code is there to help. All open source in case it's someone outside of AMD that wants to write it.
hjabird 3y ago
A challenge with this is that all current AMD GPU support in SYCL compilers (DPC++ w/ Codeplay's oneAPI for AMD GPUs, and AdaptiveCpp), is built atop of ROCm / HIP. If AMD were to move away from ROCm, they would have to adopt some other API for SYCL to use as a backend. Potentially, this could be the level zero spec from UXL foundation, but that is not the same as SYCL. However, no changing between APIs etc. will help with troublesome drivers, or issues with OS compatibility, or documentation. Those issues are orthogonal.
dagw 3y ago
I wish AMD would just drop ROCm AMD has a really terrible reputation for dropping old and introducing new APIs on a whim. To the extent that a lot of people in the field are completely burned out on AMD. Dropping yet another API after promising that this is "the one" would completely kill any chance of ever being taken seriously in the field.
slavik81 3y ago
> At work I simply gave up on our 4x Radeon Pro W6800 workstation because launching Tensorflow with more than 1 GPU would cause a kernel panic every time, and AMD engineers never offered a fix other than "reinstall Ubuntu". Unfortunately, it's not always easy to get an issue in front of the right people. The W6800 is an officially supported GPU and a kernel panic is a serious bug. I'd like to try to help get this issue properly triaged. Could you send me a link to your report?
- cherryteastain 3y ago
  Unfortunately I no longer work there so I sadly cannot give you a detailed report. What I do remember is that this was a Threadripper 5995WX system and based on what AMD reps told us at the time the issue was related to some unexpected interaction between the amdgpu driver and the PCIe implementation on the motherboard we were using.

arcanus 3y ago

SYCL is a terrible option. The performance of any application on SYCL is currently quite poor. Why drop ROCm (used on the world's largest supercomputer?) for an even more unproven product that AMD has no control over the roadmap? > Banding together with Intel to support SYCL would in my opinion Except that Intel has more control over SYCL and has repeatedly hurt AMD products with anticompetitive behavior in the past. Why would AMD permit their software to be controlled be a competitor? AMD is executing with ROCm NOW, with multiple Top500 supercomputer wins and deployment in major cloud vendors with MI300X. Yes, the software needs to improve, but the practice of throwing out software to start over is not a good strategy. AMD has far more momentum in the data center than INTC at present.

shihab 3y ago
I agree about ROCm. When it comes to comparison to SYCL, HIP is much closer to the spirit of SYCL than ROCm is. Both aim to help writing a single codebase that'll run across multiple hardwares. For now though, the trajectory of SYCL appears much more promising to me than HIP. HIP is already split in two parts for CPU and GPU, which is baffling, and neither part seems to receive much love from AMD.
pjmlp 3y ago
It hardly matters if most desktops and laptops will be running CUDA/SYCL instead.
hjabird 3y ago
> The performance of any application on SYCL is currently quite poor. SYCL can get pretty much equivalent performance in Kernels to eg. CUDA. Try looking at SYCL performance papers on Arxiv. Eg. see [1]. That isn't to say that SYCL code is optimised on every platform without tweaking - you do still need to put effort into target specific optimizations to get the best performance, like you would in the CUDA or HIP. > Why drop ROCm (used on the world's largest supercomputer?) Some of the world's largest super-computers / HPC applications do use SYCL for AMD! The application I'm most aware of for this is GROMACS. As to why? - because having 3 version of the same code using different programming APIs is a big maintenance burden. [1] https://arxiv.org/pdf/2309.09609.pdf https://arxiv.org/pdf/2309.09609.pdf
- arcanus 3y ago
  > Some of the world's largest super-computers / HPC applications do use SYCL for AMD! The application I'm most aware of for this is GROMACS. As to why? - because having 3 version of the same code using different programming APIs is a big maintenance burden. The fact that GROMACS is unwilling to drop CUDA support to stand fully behind SYCL is very telling.
  - hjabird 3y ago
    I wouldn't expect them to drop CUDA support, even if SYCL is a viable alternative: * The CUDA backend is mature, featureful, and significant effort has been invested into optimising it on Nvidia hardware. One does not simply throw away a performant, well validated HPC code! * Nvidia GPUs dominate the GPGPU market - unlike AMD's. * The SYCL backend is still is very new in comparison (they even state in the docs to pay extra attention to validation), and doesn't have Nvidia-specific optimisations yet. Why prioritise reimplementing what already exists?

cyanydeez 3y ago

hate to break it to you but with the mi300, they're clearly going in the same direction. I looked at their current compatibility chart for rocm and it's clearly just starting.

pokeypokes 3y ago

Important to understand that with these kinds of libraries, it's one thing to get everything working, and a whole other to have it all be fast. Given finite resources AMD are (rightfully) focused on the second, for a narrower set of use cases - namely big customers with big pockets, using the latest compute cards. See, for example, the world's fastest super computer. That's a whole lot more than a presentation tick box. As someone doing relatively small scale work, on gaming cards, you just aren't the target user. While these kinds of users are well represented on forums, they're a rounding error in terms of actual $. Switching to SYCL makes no sense. They need to unseat Nvidia, and specifically CUDA. The whole point of HIP is to clone CUDA, make it easy for users to switch and piggy back on Nvidias success. I used to work in this space (not at AMD lol), and generally I don't think the comments on HN are fair to AMD. Obviously ROCm has a long way to go (CUDA and libs are some truly amazing work), but it's going.

throwaway81523 3y ago
Don't know about the libs but CUDA itself is a C++ dialect targeted at a family of quirky array processors (GPU's). That takes some dev work by compiler hackers but doesn't seem amazing. Am I missing something? It's of course still easy to mess up such a project by not bringing enough clue, and AMD might have done that with ROCm. Is that what happened, or is it something different?
- pjmlp 3y ago
  CUDA is more than a C++ dialect, which is a big thing people keep missing with all those "CUDA replacements". Until CUDA 3.0, it was similar to OpenCL, a C dialect, however afterwards it became a C, C++ dialect, with common infrastructure PTX. PGI targeted PTX, with their C, C++, and very relevant, Fortran compilers for HPC. PGI was acquired by NVidia, and became the main set of CUDA compilers. Given PTX, many other languages started targeting CUDA as well, Java, .NET, Haskell, Julia, at very least. NVidia is now invested into a Python JIT for CUDA as well. So yeah, while C++20 is the main language in CUDA, there is also a whole ecosystem of programming languages, that the "CUDA replacements" keep ignoring.

kimixa 3y ago

I feel this is falling into the rewrite trap - rather than continuing to develop an existing but imperfect system, throw it all away for something that is unproven and currently in a notably worse state. What about rocm makes it fundamentally by design that much worse? And what about sycl makes it better? Rocm is already based on many standard shared OSS projects, like llvm and clang. Often the same ones SYCL stacks are based on. What about SYCL makes it so superior that it'll make up for throwing all the current work away?

imtringued 3y ago
Most likely the ability to run software on the GPU. When I last used ROCm I got both bad performance and it was prone to crashing.
- JonChesterfield 3y ago
  It's really sensitive to the (Linux) driver/compiler/library versions all lining up. If you get the blessed Ubuntu kernel version and use it with the exact corresponding driver from dkms and the matching userspace version it's reasonably solid. If you use the driver in Linux upstream, whether you have a good time or a terrible one varies with the age of the userspace libraries you're pairing with it. I don't know what to do about this setup. I think it's an artifact of gearing testing towards "releases" of the entire stack with specific HPC operating systems in mind. To do this right, each component would need to be individually solid and deal with version/interface slip relative to other components, which means the cross product of testing patterns is rapidly infeasible. I'm thinking about it but don't see a solution yet.
cherryteastain 3y ago
> What about rocm makes it fundamentally by design that much worse? And what about sycl makes it better? The fact that SYCL compiles device code to SPIR-V instead of a device specific ISA like HIP for instance. The decision to do the latter, alone, is what causes ROCm to have support for a very narrow range of cards which significantly hampers adoption.

zozbot234 3y ago

SYCL is practically comparable to OpenCL which is already getting unofficial support (including on older hardware that Rocm does not support) via Mesa's RustiCL. Additionally clspv and Mesa's Zink are both exploring OpenCL (and potentially SYCL)-on-Vulkan, though that option involves lots of quirks since the standards have evolved independently (and are even based on different varieties of SPIR-V).

JonChesterfield 3y ago

A few issues with that. One is that ROCm is the entire compute stack - driver, libraries, compilers, languages and sycl is a programming language. Sycl is more like one more language that could be built on ROCm than a replacement for it. The second is that all the entire target market wrote all their stuff in cuda and looks totally unwilling to change to anything else. That makes HIP look like a good idea and Sycl not so much. Noone (commercially significant) is likely to port their code to Sycl to benefit from Intel's GPUs. It's worth saying that AMD bet heavily on cross platform technology. Initially opencl, but take a look at the companies behind HSA as well. I'd say that doesn't appear to have worked at all - they've got the hassle of working with external specifications, and other companies have given up on the specs, so really noone is winning there. I think Triton is a cross company thing currently being implemented. Finally Intel specifically is moribund. Definitely don't want to be relying on them to do anything. On the bright side, the wide array of GPU programming languages are all very similar to each other from the perspective of the compiler stack, e.g. they all run through the same LLVM back end. So if someone decides they like sycl on amdgpu enough to write it, a lot of existing code is there to help. All open source in case it's someone outside of AMD that wants to write it.

hjabird 3y ago

A challenge with this is that all current AMD GPU support in SYCL compilers (DPC++ w/ Codeplay's oneAPI for AMD GPUs, and AdaptiveCpp), is built atop of ROCm / HIP. If AMD were to move away from ROCm, they would have to adopt some other API for SYCL to use as a backend. Potentially, this could be the level zero spec from UXL foundation, but that is not the same as SYCL. However, no changing between APIs etc. will help with troublesome drivers, or issues with OS compatibility, or documentation. Those issues are orthogonal.

dagw 3y ago

I wish AMD would just drop ROCm AMD has a really terrible reputation for dropping old and introducing new APIs on a whim. To the extent that a lot of people in the field are completely burned out on AMD. Dropping yet another API after promising that this is "the one" would completely kill any chance of ever being taken seriously in the field.

slavik81 3y ago

> At work I simply gave up on our 4x Radeon Pro W6800 workstation because launching Tensorflow with more than 1 GPU would cause a kernel panic every time, and AMD engineers never offered a fix other than "reinstall Ubuntu". Unfortunately, it's not always easy to get an issue in front of the right people. The W6800 is an officially supported GPU and a kernel panic is a serious bug. I'd like to try to help get this issue properly triaged. Could you send me a link to your report?

cherryteastain 3y ago
Unfortunately I no longer work there so I sadly cannot give you a detailed report. What I do remember is that this was a Threadripper 5995WX system and based on what AMD reps told us at the time the issue was related to some unexpected interaction between the amdgpu driver and the PCIe implementation on the motherboard we were using.