4 ms·

If you’re going to be investing much time and energy into AI/ML on your GPU, I urge you to consider going the nvidia route rather than an AMD GPU. (CPU-wise, ei

by Arubis 3y ago

If you’re going to be investing much time and energy into AI/ML on your GPU, I urge you to consider going the nvidia route rather than an AMD GPU. (CPU-wise, either AMD or Intel is just fine.) Using ROCm instead of CUDA may be possible, but it’s unloved, unsupported on consumer hardware, doesn’t even build with PyTorch at its current minor version, and generally a frustrating time-suck. The proprietary nvidia driver situation is unfortunate, and if you’re sufficiently driven off by it, ROCm can work, sometimes, eventually. But do consider the effort and time it’ll cost.

BLKNSLVR 3y ago
I think Arubis didn't push the point hard enough. Nvidia is a long way ahead of AMD in AI/ML hardware capability, from my very limited experience. I got a Stable Diffusion thing up and running locally on a relatively old AMD RX 580 with 8GB of memory, and default settings of 512x512 with 10 (I think) passes, and it takes about 10 minutes to generate an image. On a similarly aged Nvidia 1060 with 6GB of memory that a colleague of mine uses for their local Stable Diffusion instance, it takes about 20 seconds to generate an image. I don't know how apples and oranges this comparison is, my OS is Linux and his is Windows, for example, but 10 minutes versus 20 seconds is, for me, a massively disappointing difference. I'm sure the gap has decreased in the intervening 6-7 years, but I don't know how much - such that I'd be opting for a second hand Nvidia over a new AMD (and I philosophically prefer AMD). This may be a worthwhile read if you're looking to squeeze the most performance: https://timdettmers.com/2023/01/30/which-gpu-for-deep-learning/ https://timdettmers.com/2023/01/30/which-gpu-for-deep-learni...
- Aeolun 3y ago
  That doesn’t seem like a difference caused by the chip, but by whatever integration is used.
- nicopappl 3y ago
  To clarify this. On my RX5700 XT, with ROCm, Stable Diffusion generates images on the default sampler at 512x512 20 passes in about 2 seconds. It's definitively slower than a similar NVIDIA card, but I've no concrete numbers. It's still a pain in other ways. AMD does not support ROCm on consumer-grade GPUs, it only "happens to work". I had to explicitly install the 3.10 python interpreter and use an older torch version, because seemingly, the newer ROCm releases do not support the RX5700 XT. Also ROCm only works on Linux.
BeefWellington 3y ago
When you say "unsupported on consumer hardware", make it clear that it does work it just isn't supported if you run into issues. My experience with a RX6800 was that compiling ROCm for myself for use with Pytorch just worked, though from reading online it's clear that's a rarity. Current gen cards apparently don't even work.
slowmovintarget 3y ago
That's the plan. AMD CPUs and Nvidia GPUs.
- anonbanker 3y ago
  change your plans. Rusticl will make your life less painful. Avoid Nvidia until NVK matures.
anonbanker 3y ago
Now that Mesa supports rusticl, your concerns are irrelevant. ROCm is no longer needed.

by Arubis 3y ago

BLKNSLVR 3y ago

I think Arubis didn't push the point hard enough. Nvidia is a long way ahead of AMD in AI/ML hardware capability, from my very limited experience. I got a Stable Diffusion thing up and running locally on a relatively old AMD RX 580 with 8GB of memory, and default settings of 512x512 with 10 (I think) passes, and it takes about 10 minutes to generate an image. On a similarly aged Nvidia 1060 with 6GB of memory that a colleague of mine uses for their local Stable Diffusion instance, it takes about 20 seconds to generate an image. I don't know how apples and oranges this comparison is, my OS is Linux and his is Windows, for example, but 10 minutes versus 20 seconds is, for me, a massively disappointing difference. I'm sure the gap has decreased in the intervening 6-7 years, but I don't know how much - such that I'd be opting for a second hand Nvidia over a new AMD (and I philosophically prefer AMD). This may be a worthwhile read if you're looking to squeeze the most performance: https://timdettmers.com/2023/01/30/which-gpu-for-deep-learning/ https://timdettmers.com/2023/01/30/which-gpu-for-deep-learni...

Aeolun 3y ago
That doesn’t seem like a difference caused by the chip, but by whatever integration is used.
nicopappl 3y ago
To clarify this. On my RX5700 XT, with ROCm, Stable Diffusion generates images on the default sampler at 512x512 20 passes in about 2 seconds. It's definitively slower than a similar NVIDIA card, but I've no concrete numbers. It's still a pain in other ways. AMD does not support ROCm on consumer-grade GPUs, it only "happens to work". I had to explicitly install the 3.10 python interpreter and use an older torch version, because seemingly, the newer ROCm releases do not support the RX5700 XT. Also ROCm only works on Linux.

BeefWellington 3y ago

When you say "unsupported on consumer hardware", make it clear that it does work it just isn't supported if you run into issues. My experience with a RX6800 was that compiling ROCm for myself for use with Pytorch just worked, though from reading online it's clear that's a rarity. Current gen cards apparently don't even work.

slowmovintarget 3y ago

That's the plan. AMD CPUs and Nvidia GPUs.

anonbanker 3y ago
change your plans. Rusticl will make your life less painful. Avoid Nvidia until NVK matures.

anonbanker 3y ago

Now that Mesa supports rusticl, your concerns are irrelevant. ROCm is no longer needed.