3 ms·

AMD supports only a single Radeon GPU in Linux (RX 7900 in three variants)? Windows support is also bad, but supports significantly more than one GPU.

by ac29 2y ago

llm_trw 2y ago
Imagine nvidia supported only the 4090, 4080 and 4070 for cuda at the consumer level. With the 3090 not being supported since the 40xx series came out. This is what amd is defending here.
Delk 2y ago
I honestly can't figure out which Radeon GPUs are supposed to be supported. The GitHub discussion page in the title lists RX 6800 (and a bunch of RX 7xxx GPUs) as supported, and some lower-end RX 6xxx ones as supported for runtime. The same comment also links to a page on the AMD website for a "compatibility matrix" [1]. That page only shows RX 7900 variants as supported on the consumer Radeon tab. On the workstation side, Radeon Pro W6800 and some W7xxx cards are listed as supported. It also suggests to see the "Use ROCm on Radeon GPU documentation" page [2] if using ROCm on Radeon or Radeon Pro cards. That link leads to a page for "compatibility matrices" -- again. If you click the link for Linux compatibility, you get a page on "Linux support matrices by ROCm version" [3]. That "by ROCm version" page literally only has a subsection for ROCm 6.2.3. It only lists RX 7900 and Pro W7xxx cards as supported. No mention of W6800. (The page does have an unintuitively placed "Version List" link through which you can find docs for ROCm 5.7 [4]. Those older docs are no more useful than the 6.2.3 ones.) Is RX 6800 supported? Or W6800? Even the amd.com pages seem to contradict each other on the latter. Maybe the pages on the AMD site only list official production support or something. In any case it's confusing as hell. Nothing against the GitHub page author who at least seems to try and be clear but the official documentation leaves a lot to be desired. [1] https://rocm.docs.amd.com/projects/install-on-linux/en/latest/reference/system-requirements.html https://rocm.docs.amd.com/projects/install-on-linux/en/lates... [2] https://rocm.docs.amd.com/projects/radeon/en/latest/docs/compatibility.html https://rocm.docs.amd.com/projects/radeon/en/latest/docs/com... [3] https://rocm.docs.amd.com/projects/radeon/en/latest/docs/compatibility/native_linux/native_linux_compatibility.html https://rocm.docs.amd.com/projects/radeon/en/latest/docs/com... [4] https://rocm.docs.amd.com/projects/radeon/en/docs-5.7.0/docs/compatibility.html https://rocm.docs.amd.com/projects/radeon/en/docs-5.7.0/docs...
- baby_souffle 2y ago
  > I honestly can't figure out which Radeon GPUs are supposed to be supported. Exactly. I have a 6700 XT with 12 gig ram and a 5700 with 8 gig ram. If i ctrl+f for either of those numbers on the GH issue, I get one hit. For the 6700, it's a single row that has a green check for "runtime" and a red x for "HIP SDK". For the 5700 card, it's somebody in the peanut gallery saying "don't forget about us!". HIP is the c++ "flavor" that can compile down to work on amd _and_ nvidia gpus. If the 6700 has support for the "runtime" but not HIP ... what does that even mean for me? And as you pointed out, the 6800 series card has green checks for both so that means it's fully supported? But ... it's not listed on AMD's site?! Bad docs are how you cement a reputation of "just buy nvidia and install their latest drivers and it'll be fine".
  - xmodem 2y ago
    I think the matrix shown in the github issue is for Windows support, which is much better: https://rocm.docs.amd.com/projects/install-on-windows/en/latest/reference/system-requirements.html https://rocm.docs.amd.com/projects/install-on-windows/en/lat... Having said that, on the weekend I set up ROCm on Linux on my 6800XT and it seems to work just fine.
- redmajor12 2y ago
  Removing support for Radeon VII is a bonehead move that smacks of stupidity or greed. The cards were targeted for enthusiast gamers but have enterprise level hardware, like HBM2 memory and 1 TB/s bandwidth.
- magic_at_nodai 2y ago
  I will provide this feedback to the docs team to clean up. I found it hard when i was making that Poll :D but I looked harder instead of trying to fix the docs. So thank you for the feedback.
  - Delk 2y ago
    Thanks for the effort! Documentation is so easy to neglect especially when there's also a ton of other stuff to do, but it's also so important for anything intended for technical use. Especially when things change over time or from one version to another.
cokecan 2y ago
Super annoying. I have an RX 6600 XT and can't get ROCm to work on Linux. Vulkan ML however worked perfectly out of the box, so at least I got something. Just weird the official thing doesn't work.
- suprjami 2y ago
  Use the Debian libraries, it works: https://github.com/superjamie/rocswap https://github.com/superjamie/rocswap
  - slavik81 2y ago
    The caveat being that PyTorch has a lot of dependencies and a couple of them are not yet available in Debian Unstable. For folks wanting to use StableDiffusion, that's a problem. However, the available packages are more than sufficient for llama-cpp as you point out.
curt15 2y ago
I found that striking as well. Does AMD expect everyone wanting to try out PyTorch or LLMs on Linux to splurge on Instinct servers?
- magic_at_nodai 2y ago
  ROCm on Radeon should work too and the poll above was to seek feedback on what to cards to support next.

AMD supports only a single Radeon GPU in Linux (RX 7900 in three variants)? Windows support is also bad, but supports significantly more than one GPU.

by ac29 2y ago

AMD supports only a single Radeon GPU in Linux (RX 7900 in three variants)? Windows support is also bad, but supports significantly more than one GPU.

llm_trw 2y ago

Imagine nvidia supported only the 4090, 4080 and 4070 for cuda at the consumer level. With the 3090 not being supported since the 40xx series came out. This is what amd is defending here.

Delk 2y ago

I honestly can't figure out which Radeon GPUs are supposed to be supported. The GitHub discussion page in the title lists RX 6800 (and a bunch of RX 7xxx GPUs) as supported, and some lower-end RX 6xxx ones as supported for runtime. The same comment also links to a page on the AMD website for a "compatibility matrix" [1]. That page only shows RX 7900 variants as supported on the consumer Radeon tab. On the workstation side, Radeon Pro W6800 and some W7xxx cards are listed as supported. It also suggests to see the "Use ROCm on Radeon GPU documentation" page [2] if using ROCm on Radeon or Radeon Pro cards. That link leads to a page for "compatibility matrices" -- again. If you click the link for Linux compatibility, you get a page on "Linux support matrices by ROCm version" [3]. That "by ROCm version" page literally only has a subsection for ROCm 6.2.3. It only lists RX 7900 and Pro W7xxx cards as supported. No mention of W6800. (The page does have an unintuitively placed "Version List" link through which you can find docs for ROCm 5.7 [4]. Those older docs are no more useful than the 6.2.3 ones.) Is RX 6800 supported? Or W6800? Even the amd.com pages seem to contradict each other on the latter. Maybe the pages on the AMD site only list official production support or something. In any case it's confusing as hell. Nothing against the GitHub page author who at least seems to try and be clear but the official documentation leaves a lot to be desired. [1] https://rocm.docs.amd.com/projects/install-on-linux/en/latest/reference/system-requirements.html https://rocm.docs.amd.com/projects/install-on-linux/en/lates... [2] https://rocm.docs.amd.com/projects/radeon/en/latest/docs/compatibility.html https://rocm.docs.amd.com/projects/radeon/en/latest/docs/com... [3] https://rocm.docs.amd.com/projects/radeon/en/latest/docs/compatibility/native_linux/native_linux_compatibility.html https://rocm.docs.amd.com/projects/radeon/en/latest/docs/com... [4] https://rocm.docs.amd.com/projects/radeon/en/docs-5.7.0/docs/compatibility.html https://rocm.docs.amd.com/projects/radeon/en/docs-5.7.0/docs...

baby_souffle 2y ago
> I honestly can't figure out which Radeon GPUs are supposed to be supported. Exactly. I have a 6700 XT with 12 gig ram and a 5700 with 8 gig ram. If i ctrl+f for either of those numbers on the GH issue, I get one hit. For the 6700, it's a single row that has a green check for "runtime" and a red x for "HIP SDK". For the 5700 card, it's somebody in the peanut gallery saying "don't forget about us!". HIP is the c++ "flavor" that can compile down to work on amd _and_ nvidia gpus. If the 6700 has support for the "runtime" but not HIP ... what does that even mean for me? And as you pointed out, the 6800 series card has green checks for both so that means it's fully supported? But ... it's not listed on AMD's site?! Bad docs are how you cement a reputation of "just buy nvidia and install their latest drivers and it'll be fine".
- xmodem 2y ago
  I think the matrix shown in the github issue is for Windows support, which is much better: https://rocm.docs.amd.com/projects/install-on-windows/en/latest/reference/system-requirements.html https://rocm.docs.amd.com/projects/install-on-windows/en/lat... Having said that, on the weekend I set up ROCm on Linux on my 6800XT and it seems to work just fine.
redmajor12 2y ago
Removing support for Radeon VII is a bonehead move that smacks of stupidity or greed. The cards were targeted for enthusiast gamers but have enterprise level hardware, like HBM2 memory and 1 TB/s bandwidth.
magic_at_nodai 2y ago
I will provide this feedback to the docs team to clean up. I found it hard when i was making that Poll :D but I looked harder instead of trying to fix the docs. So thank you for the feedback.
- Delk 2y ago
  Thanks for the effort! Documentation is so easy to neglect especially when there's also a ton of other stuff to do, but it's also so important for anything intended for technical use. Especially when things change over time or from one version to another.

cokecan 2y ago

Super annoying. I have an RX 6600 XT and can't get ROCm to work on Linux. Vulkan ML however worked perfectly out of the box, so at least I got something. Just weird the official thing doesn't work.

suprjami 2y ago
Use the Debian libraries, it works: https://github.com/superjamie/rocswap https://github.com/superjamie/rocswap
- slavik81 2y ago
  The caveat being that PyTorch has a lot of dependencies and a couple of them are not yet available in Debian Unstable. For folks wanting to use StableDiffusion, that's a problem. However, the available packages are more than sufficient for llama-cpp as you point out.

curt15 2y ago

I found that striking as well. Does AMD expect everyone wanting to try out PyTorch or LLMs on Linux to splurge on Instinct servers?

magic_at_nodai 2y ago
ROCm on Radeon should work too and the poll above was to seek feedback on what to cards to support next.