5 ms·

(Speaking for me) Yes. It's not 500$ extra for 8GB more, it's 500$ extra for being able to do ML at all basically. AMD GPUs have near zero support in major ML

(Speaking for me) Yes. It's not 500$ extra for 8GB more, it's 500$ extra for being able to do ML at all basically. AMD GPUs have near zero support in major ML frameworks. There are some things coming out with Rocm and other niche things, but most people in ML already have enough work dealing with model and framework problems that using experimental AMD support is probably a no go. Hell, if AMD had a card with 8GB ram more than nvidia, and for 500$ cheaper, I would still go with nvidia. Everyone wish AMD would step their game up w.r.t ML workloads but it's just not happening (yet), Nvidia has a complete monopoly there.

delusional 6y ago
This is more believable to me than the RAM being the decision maker. As an ML amateur it looks like most of it is built upon proprietary nvidia tech, which makes makes AMD a nonstarter, even if they did have a technically stronger card.
reilly3000 6y ago
Would PlaidML be a viable option for you? https://github.com/plaidml/plaidml https://github.com/plaidml/plaidml
- Voloskaya 6y ago
  I don't think so: * It seems there was not a single commit in the past ~6 months which by itself is already a deal breaker. * Documentation is lackluster * You need to use Keras, I use PyTorch. This is not a deal breaker, but a significant annoyance. * Major features are still lacking. E.g. No support for quantization (afaik), which for me is fundamental. * Most importantly there seem to be no major community around it. It feels a bit bad to say that, because clearly a lot of work went into this project, and some people have to start adopting it to drive the momentum, but from an egoistical point of view, I just don't have the courage to deal with all the mess that comes with introducing an experimental layer in my workflow. Especially in ML where stuff can still appear to "work" (as in, not crashing) despite major bugs in the underlying code, leading to days or week of lost work before realizing where the issue is.
  - my123 6y ago
    PlaidML is getting a lot of work, just see the other branches. :)
- vkaku 6y ago
  I've been on plaidml w/ keras for a bit, got over most bottlenecks for now. Check out keras-helper.. I made it to switch between various backend implementations which are non NVidia specific. Pytorch may eventually need porting, but for now I don't need it. I've been trying out Coriander and DeepCL now but I decided to stick to Keras, which seems to be a decent compromise. Not using 2.4.x though, do not need it. OpenCL based backends are cutting it for me, running production workloads without needing to install CUDA/ROCm is the best way to go.
  - fluffy87 6y ago
    Why would a data scientist making >200k$/year (~1k$/ work day) spend a single second of their time trying to “workaround” something whose solution only costs 1.5k$? Spending a week/year working around ROCm would already cost you 5k$ plus the opportunity cost. For a whole team that’s a money sink.
    - exged 6y ago
      To be fair, it's not uncommon for a ML researcher / engineer to use tens (~$10/hr on cloud, $100k from Nvidia) or even hundreds (~$100/hr on cloud, $1M from Nvidia) of GPUs to speed up their iteration time. If there was a way to spend half as much on hundreds of AMD GPUs instead that would be a huge win, well worth even months of the researcher's time. The catch is that ML software stacks have had hundreds if not thousands of man-years of effort put into things like cuDNN, CUDA operator implementations, and Nvidia-specific system code (eg. for distributed training). Many formidable competitors like Google TPU have emerged, but Nvidia is currently holding onto its leadership position for now because the wide support and polish is just not there for any of the competitors yet.
    - ebalit 6y ago
      There are data scientist out of the Silicon Valley. In France, salaries for data scientist are mostly 30k€ after tax and social contributions, especially outside of Paris. 1k€ is still largely manageable on this salary but it isn't insignificant.
Twirrim 6y ago
> AMD GPUs have near zero support in major ML frameworks. There are some things coming out with Rocm and other niche things, but most people in ML already have enough work dealing with model and framework problems that using experimental AMD support is probably a no go. _If_ AMD has made a sufficiently powerful GPU, that will add a lot of incentive to ML frameworks to support it. But it's going to have to be a big difference, I imagine. Given how active AMD is in open source work, I'm a little surprised they haven't been throwing developers at ML frameworks.
- NikolaeVarius 6y ago
  Takes time and money. NVIDIA has been tossing money and PhDs at this for over a decade
- sudosysgen 6y ago
  AMD had more powerful GPUs for compute than NVidia for all but the last 2 cycles, and yet it didn't materialize.
- grp000 6y ago
  With ML workloads being so ram intensive, more than a few times I wished AMD properly supported the right ML packages. Their cost/GB is such a good deal vs Nvidia.
t-vi 6y ago
I think PyTorch support is decent, I use it on my Radeon VII a lot. - You can compile on https://lernapparat.de/pytorch-rocm/ https://lernapparat.de/pytorch-rocm/ (disclaimer my own link) - Arch has support out of the box: https://aur.archlinux.org/packages/python-pytorch-rocm/ https://aur.archlinux.org/packages/python-pytorch-rocm/ - AMD also has a ROCm docker with PyTorch (https://rocmdocs.amd.com/en/latest/Deep_learning/Deep-learning.html#recommended-install-using-published-pytorch-rocm-docker-image https://rocmdocs.amd.com/en/latest/Deep_learning/Deep-learni...) - If you know the secret URL (which you can infer from the other URLs on the front page), there are also test wheels for rocm3.8. So AMD got a lot of heat for not supporting Navi (RDNA) with ROCm, but it seems that they are weeding out the things keeping people from running it ( https://github.com/ROCmSoftwarePlatform/pytorch/issues/718 https://github.com/ROCmSoftwarePlatform/pytorch/issues/718 and the links in that look like gfx10 is almost there for rocBlas and MIOpen). We'll see what ROCm 3.9 will bring and what the state of big navi is.
- fluffything 6y ago
  I have a 5800 XT and I've just given up on ROCm support for it at this point. Why would I get a Radeon VII when used nvidia cards for machine learning are extremely cheap, and then I don't have to worry about experimental stuff breaking one day before deadline lol
- philjohn 6y ago
  Part of it is also AMD segmenting their gaming cards (RDNA) from their compute-focussed cards (CDNA).
sangnoir 6y ago
> AMD GPUs have near zero support in major ML frameworks. That is incorrect - I have been running Tensorflow on RX5*0 cards for close to 2 years now. I even transitioned to TF2 with no problem. Granted, I have to be extra careful about kernel versions, and upgrading kernels is a delicate dance involving AMD driver modules and ROCm & rocm-tensorflow. My setup is certainly finicky, but to say AMD GPUs have near zero support is false.

delusional 6y ago

This is more believable to me than the RAM being the decision maker. As an ML amateur it looks like most of it is built upon proprietary nvidia tech, which makes makes AMD a nonstarter, even if they did have a technically stronger card.

reilly3000 6y ago

Would PlaidML be a viable option for you? https://github.com/plaidml/plaidml https://github.com/plaidml/plaidml

Voloskaya 6y ago
I don't think so: * It seems there was not a single commit in the past ~6 months which by itself is already a deal breaker. * Documentation is lackluster * You need to use Keras, I use PyTorch. This is not a deal breaker, but a significant annoyance. * Major features are still lacking. E.g. No support for quantization (afaik), which for me is fundamental. * Most importantly there seem to be no major community around it. It feels a bit bad to say that, because clearly a lot of work went into this project, and some people have to start adopting it to drive the momentum, but from an egoistical point of view, I just don't have the courage to deal with all the mess that comes with introducing an experimental layer in my workflow. Especially in ML where stuff can still appear to "work" (as in, not crashing) despite major bugs in the underlying code, leading to days or week of lost work before realizing where the issue is.
- my123 6y ago
  PlaidML is getting a lot of work, just see the other branches. :)
vkaku 6y ago
I've been on plaidml w/ keras for a bit, got over most bottlenecks for now. Check out keras-helper.. I made it to switch between various backend implementations which are non NVidia specific. Pytorch may eventually need porting, but for now I don't need it. I've been trying out Coriander and DeepCL now but I decided to stick to Keras, which seems to be a decent compromise. Not using 2.4.x though, do not need it. OpenCL based backends are cutting it for me, running production workloads without needing to install CUDA/ROCm is the best way to go.
- fluffy87 6y ago
  Why would a data scientist making >200k$/year (~1k$/ work day) spend a single second of their time trying to “workaround” something whose solution only costs 1.5k$? Spending a week/year working around ROCm would already cost you 5k$ plus the opportunity cost. For a whole team that’s a money sink.
  - exged 6y ago
    To be fair, it's not uncommon for a ML researcher / engineer to use tens (~$10/hr on cloud, $100k from Nvidia) or even hundreds (~$100/hr on cloud, $1M from Nvidia) of GPUs to speed up their iteration time. If there was a way to spend half as much on hundreds of AMD GPUs instead that would be a huge win, well worth even months of the researcher's time. The catch is that ML software stacks have had hundreds if not thousands of man-years of effort put into things like cuDNN, CUDA operator implementations, and Nvidia-specific system code (eg. for distributed training). Many formidable competitors like Google TPU have emerged, but Nvidia is currently holding onto its leadership position for now because the wide support and polish is just not there for any of the competitors yet.
  - ebalit 6y ago
    There are data scientist out of the Silicon Valley. In France, salaries for data scientist are mostly 30k€ after tax and social contributions, especially outside of Paris. 1k€ is still largely manageable on this salary but it isn't insignificant.

Twirrim 6y ago

> AMD GPUs have near zero support in major ML frameworks. There are some things coming out with Rocm and other niche things, but most people in ML already have enough work dealing with model and framework problems that using experimental AMD support is probably a no go. _If_ AMD has made a sufficiently powerful GPU, that will add a lot of incentive to ML frameworks to support it. But it's going to have to be a big difference, I imagine. Given how active AMD is in open source work, I'm a little surprised they haven't been throwing developers at ML frameworks.

NikolaeVarius 6y ago
Takes time and money. NVIDIA has been tossing money and PhDs at this for over a decade
sudosysgen 6y ago
AMD had more powerful GPUs for compute than NVidia for all but the last 2 cycles, and yet it didn't materialize.
grp000 6y ago
With ML workloads being so ram intensive, more than a few times I wished AMD properly supported the right ML packages. Their cost/GB is such a good deal vs Nvidia.

t-vi 6y ago

I think PyTorch support is decent, I use it on my Radeon VII a lot. - You can compile on https://lernapparat.de/pytorch-rocm/ https://lernapparat.de/pytorch-rocm/ (disclaimer my own link) - Arch has support out of the box: https://aur.archlinux.org/packages/python-pytorch-rocm/ https://aur.archlinux.org/packages/python-pytorch-rocm/ - AMD also has a ROCm docker with PyTorch (https://rocmdocs.amd.com/en/latest/Deep_learning/Deep-learning.html#recommended-install-using-published-pytorch-rocm-docker-image https://rocmdocs.amd.com/en/latest/Deep_learning/Deep-learni...) - If you know the secret URL (which you can infer from the other URLs on the front page), there are also test wheels for rocm3.8. So AMD got a lot of heat for not supporting Navi (RDNA) with ROCm, but it seems that they are weeding out the things keeping people from running it ( https://github.com/ROCmSoftwarePlatform/pytorch/issues/718 https://github.com/ROCmSoftwarePlatform/pytorch/issues/718 and the links in that look like gfx10 is almost there for rocBlas and MIOpen). We'll see what ROCm 3.9 will bring and what the state of big navi is.

fluffything 6y ago
I have a 5800 XT and I've just given up on ROCm support for it at this point. Why would I get a Radeon VII when used nvidia cards for machine learning are extremely cheap, and then I don't have to worry about experimental stuff breaking one day before deadline lol
philjohn 6y ago
Part of it is also AMD segmenting their gaming cards (RDNA) from their compute-focussed cards (CDNA).

sangnoir 6y ago

> AMD GPUs have near zero support in major ML frameworks. That is incorrect - I have been running Tensorflow on RX5*0 cards for close to 2 years now. I even transitioned to TF2 with no problem. Granted, I have to be extra careful about kernel versions, and upgrading kernels is a delicate dance involving AMD driver modules and ROCm & rocm-tensorflow. My setup is certainly finicky, but to say AMD GPUs have near zero support is false.