6 ms·

AMD ROCm Software Blogs

118 points by jrepinc 3y ago

brucethemoose2 3y ago
Some of these are pretty good. The AITemplate Stable diffusion demo, for instance, is a nice hidden gem (though I'm not sure if it works on consumer hardware these days).
mcbuilder 3y ago
With a little finagling, I was able to get ComfyUI working for my AMD Cards. I purchased 7800XT with 16GB RAM and have been pretty happy with its value. Getting around 9it/s for a simple SD 1.5 pipeline using 512x512 latent image. Not a speed beast, but plenty fast to teach my kids about AI and run some local models.
- tempest_ 3y ago
  I have automatic1111 working on my 7900xtx and it does alright. Took some finagling to get it working with ROCM 6 and the newest pytorch which I couldnt seem to coax invoke or comfy to do
- jauntywundrkind 3y ago
  The 7900 line-up (and the old VII weirdly) are the only supported consumer cards that do ROCm 6. https://rocm.docs.amd.com/projects/install-on-linux/en/latest/reference/system-requirements.html https://rocm.docs.amd.com/projects/install-on-linux/en/lates... There seems to be hit and miss reports about coaxing a 7800XT to run. But the uncertainty (and the fact that it's no better/in some ways worse than a 6800xt) basically dissuaded me from bothering to upgrade my old rx580 recently.
- weebull 3y ago
  I've done similar with an 8GB 7600, and I've run SD1.5, SDXL and Stable Cascade on it with ComfyUI. To get beat speeds I need to not run the desktop. That way I get all 8GB for inferredce, and control it from another computer. I think the 16GB 7600XT could be a nice entry level card for ML applications.
mikeweiss 3y ago
When are we likely to see MI300 available for rent and what would the cost per hour be?
- photonbucket 3y ago
  Which companies that already rent out H100s would risk the ire of nvidia providing them less capacity next time? edit: It seems cudocompute and dataknox is willing to rent them out
- latchkey 3y ago
  Thanks for asking. Soon! Estimated ship date is 2/29. Should arrive around 5 days after that. We are going to fly to the data center, get the boxes all set up and start to launch things. Pricing is TBD since there isn't really a comparable and we expect that since these GPUs are so new, people will want to do a lot of testing before they are willing to commit. Therefore, we are looking for partnerships over just customers. Disclosure: Check out my profile, I'm building a white glove bare metal service that is focused on only top end AMD compute and specifically the MI300x (and future generations). Feel free to reach out via email.
  - brucethemoose2 3y ago
    We can't quite afford an 8x bare metal instance, but I am pinning this for later. Think ya'll will ever offer single MI300X instances?
    - latchkey 3y ago
      Yes! Like I said above, we fully expect people to be toe dipping at first given how new this hardware is. We offer single GPU access through a VM and PCIe pass through. Either we can provide you a basic Ubuntu VM with everything loaded into it (with regards to drivers and such) or you can send us your VM image and set it up however you like. Then you'll just have direct ssh access into it.
      - 3y ago
hiddencost 3y ago
Waiting for this to work on windows. My gaming machine is a windows machine, so that's where my GPU is. Not willing to add a Linux partition, and the "Linux in Windows" support doesn't work with AMD gpus.
- Aaronstotle 3y ago
  You could add another ssd and have that be a linux distro. Its not a partition per se, since the whole drive is dedicated to it. I have that on my PC and I like it a lot
  - smallmancontrov 3y ago
    This is the way. Using a partition is just begging for an update (Windows or Linux) to kill your boot. Convert the amount of time you would spend periodically fixing a broken boot into a dollar amount and suddenly having a separate SSD looks dirt cheap. See also: fixing software bluetooth issues, especially those caused by dual boot, by using an external 3.5mm or toslink -> bluetooth transmitter. I use a B03Pro and haven't had to fight bluetooth pairing / quality / microphone issues in years, despite dual booting. Software sucks. Replace it with dedicated hardware and your life will suck less.
    - tombert 3y ago
      > Using a partition is just begging for an update (Windows or Linux) to kill your boot. Everyone says that, but I really haven't found that to be true in the last 12 years or so. I dual-booted Windows and Arch Linux for multiple years (2014-2017), doing plenty of system updates. I never really had any boot issues caused by an update. The only time I ever had any boot issues is when I was mucking with Grub configs and I didn't really know what I was doing. Maybe I've just been lucky, but I've had a ton of computers, many of which I've dual-booted with partitions, and it feels like a pretty streamlined process that doesn't seem to break.
      - smallmancontrov 3y ago
vagab0nd 3y ago
Looks like tinycorp is shipping a box with 6x 7900 soon. https://twitter.com/__tinygrad__/status/1760988080754856210 https://twitter.com/__tinygrad__/status/1760988080754856210
- jauntywundrkind 3y ago
  I wonder what the PCIe AER difficulties he refers to are. Switching from HIP to HSA is an interesting turn of events. I thought HSA was dead or mostly a buzzword. The Wikipedia article is short on dates, but it feels like the HSA was trying to be a thing a decade ago. https://en.wikipedia.org/wiki/Heterogeneous_System_Architecture https://en.wikipedia.org/wiki/Heterogeneous_System_Architect...
- WaxProlix 3y ago
  6x 7900 XTX according to the link. That's a lot of horsepower.
- Jackson__ 3y ago
  I can't help but wonder what'll happen to those boxes when AMD inevitably discontinues ROCm support for the 7900XTX 3 years down the line.
superkuh 3y ago
Just watch out. Sometimes AMD's ROCm support for their consumer GPU can be only 4 years (like the RX 580). So if you're not buying cutting edge on release day that means you have ~2-3 years max of ROCm support. And then you have to use opencl and let me assure you: it's not fun or widespread.
- kkielhofner 3y ago
  It also took AMD a year to officially support ROCm with the 7900xtx. IME it's still rough because ROCm is fundamentally rough but non-officially supported cards are even rougher.
- 3abiton 3y ago
  I have wasted so many nights debugging and trying to figure out how to get LLMs to run on my AMD iGPU. Despite pytorch and xformers (unofficially) being supported, I was still unsuccessful. Cuda on the other hand, it just works. I don't see the value proposition of AMD over Nvidia for now
  - superkuh 3y ago
    For that I'd suggest using llama.cpp and the clBLAST (opencl) config to compile it.
  - sam_goody 3y ago
    You could also run CUDA on AMD cards using ZLUDA.
    - 3abiton 3y ago
      ZLUDA has its own baggage for now. Still lacking many support, but that changing fast, maybe in a month or so.
montebicyclelo 3y ago
Guessing there are a few of us in a position where we are frustrated with our past experiences with ROCm software, (e.g. being awful to install, in the past there were long guides with loads of steps to follow, and not the greatest clarity / simplicity in the instructions, and the only option they suggested if you messed it up was to reinstall the operating system, and there were multiple guides / pages and it wasn't clear which was the latest one, and then it only supported older versions of tensorflow / pytorch / jax, and only supported recent / higher end cards, etc. — it may be better now, this is my experience from I guess a few years ago); but who at the same time recognise that it would be great for GPU compute to be more affordable, and for there to be good competitors to Nvidia.
- latchkey 3y ago
  I don't think that anyone is trying to whitewash the past. All we can do is look to the future. Fact is that the demand for AI, has forced their hand. As I see it, AMD is trying to clean up their act as quickly as possible, but change of this magnitude, isn't going to happen over night. These sorts of updates to things, like their blogs, are baby steps in the right direction.
  - montebicyclelo 3y ago
    Thanks, I do agree with this, and I am rooting for them. Would love to find out at some point that it's turned around.
  - hackerlight 3y ago
    Andrew Ng says ROCm has improved a lot in the last year and is no longer as bad as people make it out to be
vanillax 3y ago
https://forum.level1techs.com/t/mi25-stable-diffusions-100-hidden-beast/194172 https://forum.level1techs.com/t/mi25-stable-diffusions-100-h... https://www.youtube.com/watch?v=t4J_KYp0NGM https://www.youtube.com/watch?v=t4J_KYp0NGM interesting stuff with mi25
- latchkey 3y ago
  MI25 is dropped from ROCm supported list from 4.5.x onwards.
dang 3y ago
Lists don't make great HN submissions: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sort=byDate&type=comment&query=denominator%20lists%20by:dang https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor.... It would be better to pick the most interesting element in the list and submit that instead.
- latchkey 3y ago
  hi dang, I think in most cases you are correct, however in this case it is subtly acknowledging the general change that AMD is making with regards to overall their focus and support of ROCm (which has been a complaint in the past), more than any single blog post on the topic. I think that is what is spawning the discussion here now.
Lichtso 3y ago
If you think about doing something using ROCm, read these first to get a feeling for what to expect: https://www.reddit.com/r/Amd/comments/a9tjge/amd_rocm_hcc_programming_introduction/ https://www.reddit.com/r/Amd/comments/a9tjge/amd_rocm_hcc_pr... https://www.reddit.com/r/ROCm/ https://www.reddit.com/r/ROCm/
niij 3y ago
I have a 6950XT. Last I read it was impossible to run stable diffusion on it on windows with any decent iteration speed. Is there anywhere to track progress on that? I'm trying to practice with some AI/ML tools, but the software issues make me wish I bought a NVIDIA card instead.

118 points by jrepinc 3y ago

Some of these are pretty good. The AITemplate Stable diffusion demo, for instance, is a nice hidden gem (though I'm not sure if it works on consumer hardware these days).

mcbuilder 3y ago

With a little finagling, I was able to get ComfyUI working for my AMD Cards. I purchased 7800XT with 16GB RAM and have been pretty happy with its value. Getting around 9it/s for a simple SD 1.5 pipeline using 512x512 latent image. Not a speed beast, but plenty fast to teach my kids about AI and run some local models.

tempest_ 3y ago
I have automatic1111 working on my 7900xtx and it does alright. Took some finagling to get it working with ROCM 6 and the newest pytorch which I couldnt seem to coax invoke or comfy to do
jauntywundrkind 3y ago
The 7900 line-up (and the old VII weirdly) are the only supported consumer cards that do ROCm 6. https://rocm.docs.amd.com/projects/install-on-linux/en/latest/reference/system-requirements.html https://rocm.docs.amd.com/projects/install-on-linux/en/lates... There seems to be hit and miss reports about coaxing a 7800XT to run. But the uncertainty (and the fact that it's no better/in some ways worse than a 6800xt) basically dissuaded me from bothering to upgrade my old rx580 recently.
weebull 3y ago
I've done similar with an 8GB 7600, and I've run SD1.5, SDXL and Stable Cascade on it with ComfyUI. To get beat speeds I need to not run the desktop. That way I get all 8GB for inferredce, and control it from another computer. I think the 16GB 7600XT could be a nice entry level card for ML applications.

mikeweiss 3y ago

When are we likely to see MI300 available for rent and what would the cost per hour be?

photonbucket 3y ago
Which companies that already rent out H100s would risk the ire of nvidia providing them less capacity next time? edit: It seems cudocompute and dataknox is willing to rent them out
latchkey 3y ago
Thanks for asking. Soon! Estimated ship date is 2/29. Should arrive around 5 days after that. We are going to fly to the data center, get the boxes all set up and start to launch things. Pricing is TBD since there isn't really a comparable and we expect that since these GPUs are so new, people will want to do a lot of testing before they are willing to commit. Therefore, we are looking for partnerships over just customers. Disclosure: Check out my profile, I'm building a white glove bare metal service that is focused on only top end AMD compute and specifically the MI300x (and future generations). Feel free to reach out via email.
- brucethemoose2 3y ago
  We can't quite afford an 8x bare metal instance, but I am pinning this for later. Think ya'll will ever offer single MI300X instances?
  - latchkey 3y ago
    Yes! Like I said above, we fully expect people to be toe dipping at first given how new this hardware is. We offer single GPU access through a VM and PCIe pass through. Either we can provide you a basic Ubuntu VM with everything loaded into it (with regards to drivers and such) or you can send us your VM image and set it up however you like. Then you'll just have direct ssh access into it.
    - 3y ago

hiddencost 3y ago

Waiting for this to work on windows. My gaming machine is a windows machine, so that's where my GPU is. Not willing to add a Linux partition, and the "Linux in Windows" support doesn't work with AMD gpus.

Aaronstotle 3y ago
You could add another ssd and have that be a linux distro. Its not a partition per se, since the whole drive is dedicated to it. I have that on my PC and I like it a lot
- smallmancontrov 3y ago
  This is the way. Using a partition is just begging for an update (Windows or Linux) to kill your boot. Convert the amount of time you would spend periodically fixing a broken boot into a dollar amount and suddenly having a separate SSD looks dirt cheap. See also: fixing software bluetooth issues, especially those caused by dual boot, by using an external 3.5mm or toslink -> bluetooth transmitter. I use a B03Pro and haven't had to fight bluetooth pairing / quality / microphone issues in years, despite dual booting. Software sucks. Replace it with dedicated hardware and your life will suck less.
  - tombert 3y ago
    > Using a partition is just begging for an update (Windows or Linux) to kill your boot. Everyone says that, but I really haven't found that to be true in the last 12 years or so. I dual-booted Windows and Arch Linux for multiple years (2014-2017), doing plenty of system updates. I never really had any boot issues caused by an update. The only time I ever had any boot issues is when I was mucking with Grub configs and I didn't really know what I was doing. Maybe I've just been lucky, but I've had a ton of computers, many of which I've dual-booted with partitions, and it feels like a pretty streamlined process that doesn't seem to break.
    - smallmancontrov 3y ago

vagab0nd 3y ago

Looks like tinycorp is shipping a box with 6x 7900 soon. https://twitter.com/__tinygrad__/status/1760988080754856210 https://twitter.com/__tinygrad__/status/1760988080754856210

jauntywundrkind 3y ago
I wonder what the PCIe AER difficulties he refers to are. Switching from HIP to HSA is an interesting turn of events. I thought HSA was dead or mostly a buzzword. The Wikipedia article is short on dates, but it feels like the HSA was trying to be a thing a decade ago. https://en.wikipedia.org/wiki/Heterogeneous_System_Architecture https://en.wikipedia.org/wiki/Heterogeneous_System_Architect...
WaxProlix 3y ago
6x 7900 XTX according to the link. That's a lot of horsepower.
Jackson__ 3y ago
I can't help but wonder what'll happen to those boxes when AMD inevitably discontinues ROCm support for the 7900XTX 3 years down the line.

superkuh 3y ago

Just watch out. Sometimes AMD's ROCm support for their consumer GPU can be only 4 years (like the RX 580). So if you're not buying cutting edge on release day that means you have ~2-3 years max of ROCm support. And then you have to use opencl and let me assure you: it's not fun or widespread.

kkielhofner 3y ago
It also took AMD a year to officially support ROCm with the 7900xtx. IME it's still rough because ROCm is fundamentally rough but non-officially supported cards are even rougher.
3abiton 3y ago
I have wasted so many nights debugging and trying to figure out how to get LLMs to run on my AMD iGPU. Despite pytorch and xformers (unofficially) being supported, I was still unsuccessful. Cuda on the other hand, it just works. I don't see the value proposition of AMD over Nvidia for now
- superkuh 3y ago
  For that I'd suggest using llama.cpp and the clBLAST (opencl) config to compile it.
- sam_goody 3y ago
  You could also run CUDA on AMD cards using ZLUDA.
  - 3abiton 3y ago
    ZLUDA has its own baggage for now. Still lacking many support, but that changing fast, maybe in a month or so.

montebicyclelo 3y ago

Guessing there are a few of us in a position where we are frustrated with our past experiences with ROCm software, (e.g. being awful to install, in the past there were long guides with loads of steps to follow, and not the greatest clarity / simplicity in the instructions, and the only option they suggested if you messed it up was to reinstall the operating system, and there were multiple guides / pages and it wasn't clear which was the latest one, and then it only supported older versions of tensorflow / pytorch / jax, and only supported recent / higher end cards, etc. — it may be better now, this is my experience from I guess a few years ago); but who at the same time recognise that it would be great for GPU compute to be more affordable, and for there to be good competitors to Nvidia.

latchkey 3y ago
I don't think that anyone is trying to whitewash the past. All we can do is look to the future. Fact is that the demand for AI, has forced their hand. As I see it, AMD is trying to clean up their act as quickly as possible, but change of this magnitude, isn't going to happen over night. These sorts of updates to things, like their blogs, are baby steps in the right direction.
- montebicyclelo 3y ago
  Thanks, I do agree with this, and I am rooting for them. Would love to find out at some point that it's turned around.
- hackerlight 3y ago
  Andrew Ng says ROCm has improved a lot in the last year and is no longer as bad as people make it out to be

vanillax 3y ago

https://forum.level1techs.com/t/mi25-stable-diffusions-100-hidden-beast/194172 https://forum.level1techs.com/t/mi25-stable-diffusions-100-h... https://www.youtube.com/watch?v=t4J_KYp0NGM https://www.youtube.com/watch?v=t4J_KYp0NGM interesting stuff with mi25

latchkey 3y ago
MI25 is dropped from ROCm supported list from 4.5.x onwards.

dang 3y ago

Lists don't make great HN submissions: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sort=byDate&type=comment&query=denominator%20lists%20by:dang https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor.... It would be better to pick the most interesting element in the list and submit that instead.

latchkey 3y ago
hi dang, I think in most cases you are correct, however in this case it is subtly acknowledging the general change that AMD is making with regards to overall their focus and support of ROCm (which has been a complaint in the past), more than any single blog post on the topic. I think that is what is spawning the discussion here now.

Lichtso 3y ago

If you think about doing something using ROCm, read these first to get a feeling for what to expect: https://www.reddit.com/r/Amd/comments/a9tjge/amd_rocm_hcc_programming_introduction/ https://www.reddit.com/r/Amd/comments/a9tjge/amd_rocm_hcc_pr... https://www.reddit.com/r/ROCm/ https://www.reddit.com/r/ROCm/

niij 3y ago

I have a 6950XT. Last I read it was impossible to run stable diffusion on it on windows with any decent iteration speed. Is there anywhere to track progress on that? I'm trying to practice with some AI/ML tools, but the software issues make me wish I bought a NVIDIA card instead.