5 ms·

I agree, but the lack of usable ML acceleration is a deal breaker for me. Especially nowadays that stable diffusion and similar models can be run on your Deskto

I agree, but the lack of usable ML acceleration is a deal breaker for me. Especially nowadays that stable diffusion and similar models can be run on your Desktop. So, this gap will not only drive hobbysts/professionals away from AMD, but normal users more and more. Last time I checked ROCM is light years away from CUDA.

bayindirh 4y ago
I have my hopes high because of two reasons. 1. World's most powerful supercomputers run on AMD Radeon Instinct cards, which means more funding for better ML stack from AMD. 2. A ton of ROCm libraries developed by AMD has landed recently in Debian, which means there's actual and fast paced development work being done on this stack.
- MintsJohn 4y ago
  There's been this big promise of rocm for ages, since it hasn't taken up ever since, I see no reason for it to do so shortly (it will eventually). Yes rdna3 has ml optimizations, sure there is even some non public windows rocm build, but even if everything works out, taking into account the time the leading frameworks need to either fully use rocm at all or even update a to newer version, now is not the time to get an amd gpu when you want to do ml. Given AMDs trackrecord of not delivering workable ml solutions, it only makes sense to do so when the ecosystem is there and stable for a year or so.
- _hl_ 4y ago
  But this is all for the enterprise grade hardware. There seems to be very little interest from AMD to extend their stack to consumer GPUs. From an outside perspective I really cannot understand their reasoning here.
  - JonChesterfield 4y ago
    If you manage to build the stack from source (or your distro did) it's likely to run fine on consumer hardware. A lot of it is developed on them. I don't do ML or try to build 'real' applications though, so ymmv. And outside of the blessed rocm binaries the company line is unsupported. Kind of has to be, they weren't tested by amd. Source: I work on the llvm part of the stack and don't have enterprise hardware. I don't know what the library developers use but expect some of them have similar setups.
roenxi 4y ago
I remain a fanatically committed AMD buyer for their linux support, but wow the cost of it in the machine learning space has been painful. It really hurts when all the tutorials are for Nvidia and nothing seems to work. I suspect the ROCm situation has not improved since last time I checked, based on their documentation. The installation guide [0] is a bit of a mess. It looks like they are targeting ROCm at existing ROCm users rather than finding ways to convince people to become ROCm users; nothing there is useful introductory material. The opening is basically marketing babble too. Or possibly someone with a history in the GNU projects lineage of terrible recursive names. It just doesn't look like they've got a leadership figure trying to bring people in to their ecosystem right now. I dunno, maybe the API isn't intended for interested developers. [0] https://docs.amd.com/bundle/ROCm-Installation-Guide-v5.4/page/Introduction_to_ROCm_Installation_Guide_for_Linux.html https://docs.amd.com/bundle/ROCm-Installation-Guide-v5.4/pag... - opens with "ROCm is a brand name for the ROCm open software platform (for software) or the ROCm open platform ecosystem (includes hardware like FPGAs or other CPU architectures)".
- rerx 4y ago
  The tooling is there to convert existing CUDA code into a form that runs on AMD GPUs as well: https://docs.amd.com/en-US/bundle/HIPify-Reference-Guide-v5.1/page/HIPify.html https://docs.amd.com/en-US/bundle/HIPify-Reference-Guide-v5.... They have drop-in replacements for most Nvidia-specific libraries like cuDNN, too. So, while the documentation still doesn't make it trivial to get started, the software side definitely seems to have improved in recent years.
- Cloudef 4y ago
  It's unfortunate. CUDA is a mess, but because it's the leading tech on the field, AMD has no choice but to come with an (equally messy) compatible solution. Vulkan was thought of as a something that could replace CUDA and (dead) OpenCL, but it's never going to take off due to the ML field being heavily CUDA, or well more of pytorch / python :)
  - eyegor 4y ago
    I haven't tried it myself, but pytorch has supported amd gpus via rocm since 2021. https://www.amd.com/en/technologies/infinity-hub/pytorch https://www.amd.com/en/technologies/infinity-hub/pytorch
sorenjan 4y ago
I hope we'll see better inference solutions soon. Instead of installing some version of pytorch and downloading .pt-files I'd like to have an inference engine optimized for my computer, and be able to download trained models that run on multiple different systems. There's already DirectML on Windows, CoreML for Apple, ONNX runtime, etc. Not everyone need to train models, and maybe some fine tuning for things like Dreambooth can even be done on CPU. I think what's holding this back is lack of interest from the academics publishing the models, which is understandable. With a better inference standard option hopefully it would run on AMD too. But I also hope AMD gets their compute to a better place, preferable using open standards so we don't need "Nvidia code", "AMD code", etc.
brokenmachine 4y ago
Forgive me if this is a silly question, but why would a normal user need ML? I'm a techie and I don't think I have a use for it.
atq2119 4y ago
Since you mention stable diffusion, there's this: https://github.com/nod-ai/SHARK/blob/main/shark/examples/shark_inference/stable_diffusion/stable_diffusion_amd.md https://github.com/nod-ai/SHARK/blob/main/shark/examples/sha... Looks like it's still early days, but clearly it has recently been made to work and there's an effort going on to make it better.

bayindirh 4y ago

I have my hopes high because of two reasons. 1. World's most powerful supercomputers run on AMD Radeon Instinct cards, which means more funding for better ML stack from AMD. 2. A ton of ROCm libraries developed by AMD has landed recently in Debian, which means there's actual and fast paced development work being done on this stack.

MintsJohn 4y ago
There's been this big promise of rocm for ages, since it hasn't taken up ever since, I see no reason for it to do so shortly (it will eventually). Yes rdna3 has ml optimizations, sure there is even some non public windows rocm build, but even if everything works out, taking into account the time the leading frameworks need to either fully use rocm at all or even update a to newer version, now is not the time to get an amd gpu when you want to do ml. Given AMDs trackrecord of not delivering workable ml solutions, it only makes sense to do so when the ecosystem is there and stable for a year or so.
_hl_ 4y ago
But this is all for the enterprise grade hardware. There seems to be very little interest from AMD to extend their stack to consumer GPUs. From an outside perspective I really cannot understand their reasoning here.
- JonChesterfield 4y ago
  If you manage to build the stack from source (or your distro did) it's likely to run fine on consumer hardware. A lot of it is developed on them. I don't do ML or try to build 'real' applications though, so ymmv. And outside of the blessed rocm binaries the company line is unsupported. Kind of has to be, they weren't tested by amd. Source: I work on the llvm part of the stack and don't have enterprise hardware. I don't know what the library developers use but expect some of them have similar setups.

roenxi 4y ago

I remain a fanatically committed AMD buyer for their linux support, but wow the cost of it in the machine learning space has been painful. It really hurts when all the tutorials are for Nvidia and nothing seems to work. I suspect the ROCm situation has not improved since last time I checked, based on their documentation. The installation guide [0] is a bit of a mess. It looks like they are targeting ROCm at existing ROCm users rather than finding ways to convince people to become ROCm users; nothing there is useful introductory material. The opening is basically marketing babble too. Or possibly someone with a history in the GNU projects lineage of terrible recursive names. It just doesn't look like they've got a leadership figure trying to bring people in to their ecosystem right now. I dunno, maybe the API isn't intended for interested developers. [0] https://docs.amd.com/bundle/ROCm-Installation-Guide-v5.4/page/Introduction_to_ROCm_Installation_Guide_for_Linux.html https://docs.amd.com/bundle/ROCm-Installation-Guide-v5.4/pag... - opens with "ROCm is a brand name for the ROCm open software platform (for software) or the ROCm open platform ecosystem (includes hardware like FPGAs or other CPU architectures)".

rerx 4y ago
The tooling is there to convert existing CUDA code into a form that runs on AMD GPUs as well: https://docs.amd.com/en-US/bundle/HIPify-Reference-Guide-v5.1/page/HIPify.html https://docs.amd.com/en-US/bundle/HIPify-Reference-Guide-v5.... They have drop-in replacements for most Nvidia-specific libraries like cuDNN, too. So, while the documentation still doesn't make it trivial to get started, the software side definitely seems to have improved in recent years.
Cloudef 4y ago
It's unfortunate. CUDA is a mess, but because it's the leading tech on the field, AMD has no choice but to come with an (equally messy) compatible solution. Vulkan was thought of as a something that could replace CUDA and (dead) OpenCL, but it's never going to take off due to the ML field being heavily CUDA, or well more of pytorch / python :)
- eyegor 4y ago
  I haven't tried it myself, but pytorch has supported amd gpus via rocm since 2021. https://www.amd.com/en/technologies/infinity-hub/pytorch https://www.amd.com/en/technologies/infinity-hub/pytorch

sorenjan 4y ago

I hope we'll see better inference solutions soon. Instead of installing some version of pytorch and downloading .pt-files I'd like to have an inference engine optimized for my computer, and be able to download trained models that run on multiple different systems. There's already DirectML on Windows, CoreML for Apple, ONNX runtime, etc. Not everyone need to train models, and maybe some fine tuning for things like Dreambooth can even be done on CPU. I think what's holding this back is lack of interest from the academics publishing the models, which is understandable. With a better inference standard option hopefully it would run on AMD too. But I also hope AMD gets their compute to a better place, preferable using open standards so we don't need "Nvidia code", "AMD code", etc.

brokenmachine 4y ago

Forgive me if this is a silly question, but why would a normal user need ML? I'm a techie and I don't think I have a use for it.

atq2119 4y ago

Since you mention stable diffusion, there's this: https://github.com/nod-ai/SHARK/blob/main/shark/examples/shark_inference/stable_diffusion/stable_diffusion_amd.md https://github.com/nod-ai/SHARK/blob/main/shark/examples/sha... Looks like it's still early days, but clearly it has recently been made to work and there's an effort going on to make it better.