5 ms·

This is inference only. AMD should invest into the full AI stack starting from training. For this they need a product comparable to NVIDIA 4090, so that entry l

This is inference only. AMD should invest into the full AI stack starting from training. For this they need a product comparable to NVIDIA 4090, so that entry level researchers could use their hardware. Honestly, I don't know why AMD aren't doing that already, they are best positioned to do that in the industry landscape.

Hooray_Darakian 4y ago
> AMD should invest into the full AI stack starting from training. https://www.amd.com/en/graphics/servers-solutions-rocm-ml https://www.amd.com/en/graphics/servers-solutions-rocm-ml > For this they need a product comparable to NVIDIA 4090, so that entry level researchers could use their hardware. Why is a high end product a requirement for entry level research?
- jzwinck 4y ago
  Because high end research uses a fleet of them, not just one.
- kombine 4y ago
  4090 (or 3090, 1080Ti and so on) is a high-end consumer GPU, but at the same time it is an entry level GPU for AI researchers. Don't forget that workstation cards (RTX 8000) let alone server-grade GPUs such as A100 are an order of magnitude more expensive.
  - gymbeaux 4y ago
    I was doing ML stuff on a GTX 1060 a few years ago. As with everything, it depends on what you’re doing.
    - p1esk 4y ago
      Chances of you publishing something in ML improve proportionally to the amount of hardware you have access to. Or to put it another way, the less hardware you have, the smarter you have to be to publish something in ML.
  - DeepYogurt 4y ago
    That does not sound like "entry level" research to me.
    - UncleEntity 4y ago
      IDK, I want to train up some AI and have the choice of using google colab or buying a GPU that can do it within a reasonable timeframe. Not going to be spending $10k like the tortoise-tts guy (was looking into that project last night) but $2k might be doable for a hobby project. Plus I’d have a computer at the end.
- marcyb5st 4y ago
  Because of VRAM. Even a simple model for language can easily max out a 4090. Also, ROC-M is a bit of a mess to setup. With Nvidia i just need to install cuda, cudnn and then pip install tensorflow/pytorch.
wmf 4y ago
MI200, MI250, and MI300 should work for training. They don't have an exact equivalent to the 4090 but that may be setting the bar too high. Nobody can deliver everything that Nvidia has but better.
- kombine 4y ago
  It will be harder for a small academic lab to afford MI300, whereas everyone can purchase a few cards costing $1500. And even if I had money to buy MI300, I wouldn't - it is a too risky investment, because we have no idea how suitable they are for common AI research workflows. They need to lower the entry bar, so that people can try out their hardware. Even 80% of the performance of 4090 would be enough at an appropriate price point.
modeless 4y ago
The hardware is not their major problem. They have been failing super hard at the software side of machine learning for a solid decade now. It seems like pure management incompetence to me. They need to invest a whole lot more in software, integrating their stuff directly into pytorch/TF/XLA/etc and making sure it works on consumer cards too. The investment would be paid back tenfold. The market is crying out for more competition for Nvidia and there's huge money to be made on the datacenter side but it all needs to work on the consumer side too.
- hedgehog 4y ago
  Last I checked they see deep learning training as a niche market, their strategy is to try to win big contracts (HPC etc) and then supply software specifically for that. Then "the community" will supply software. Having spent a bunch of time beating my head on this and related walls it's not clear to me that they're entirely wrong from an economic standpoint. Remember that 2/3 public cloud providers have their own chips as well as NVIDIA's so it would be tough to negotiate a good deal. As a user it's super irritating to be stuck on NVIDIA especially when Jensen gets up on stage to say "haha, Moore's law is over, stop expecting our products to get cheaper."
  - marcyb5st 4y ago
    I hope they change their minds. At least now that generative models are becoming somewhat popular. I'd love to be able to get an AMD card to run generative models, but to the best of my knowledge, they only run on Nvidia hardware
    - nl 4y ago
      No personal experience, but you can actually get Stable Diffusion to run on AMD cards. It uses DirectML on Windows: https://gist.github.com/averad/256c507baa3dcc9464203dc14610d674 https://gist.github.com/averad/256c507baa3dcc9464203dc14610d... This is thanks to Microsoft, not AMD. On Linux you can use ROCm: https://www.videogames.ai/2022/11/06/Stable-Diffusion-AMD-GPU-ROCm-Linux.html https://www.videogames.ai/2022/11/06/Stable-Diffusion-AMD-GP... The horrible install processes and what a mess this is is all down to AMD.
hedgehog 4y ago
Big question is why? Would competing for entry level researchers buy them much?
- deleted 4y ago
  [deleted]
- kombine 4y ago
  Because entry level researchers shape the industry in the long term. I'm in academia, I worked at two universities and I have not seen a lab that uses non-NVIDIA hardware for research. Majority of graduates go to work in the industry.
  - hedgehog 4y ago
    I used to believe this idea that the tools in academia would carry over to industry. Now I think it's only weakly true, that is there's not really a big barrier to switching to other options like TPU or maybe Trainium (haven't tried it). Supporting independent researchers gives you the Heroku problem, they may like your product but as they get more sophisticated and go to production they'll accept significant pain to save money, scale better, etc. You basically have to re-win that business from scratch and the technical tradeoffs are very different at that point.
dathinab 4y ago
> product comparable to NVIDIA 4090 no, they need a product good at training and gpu compute at a reasonable price that product doesn't need to be good at rendering, ray tracing and similar sure students and some independent contractors probably love getting both a good graphic card and a CUDA card in one and it makes it easier for people to experiment with it but company PCs normally ban playing games on company PCs and the overlap of "needing max GPU compute" and "needing complicated 3D rendering tasks" is limited. through having 1 product instead of two does make supply chain and pricing easier but then 4090 is by now in a price range where students are unlikely to afford it and people will think twice about buying it just to play around with GPU compute. So e.g. the 7900XTX having somewhat comparable GPU compute usability then a 4080 would have been good enough for the non company use case, where a dedicated compute-per-money cheaper GPU compute only card would be preferable for the company use case I think.
- nl 4y ago
  Long time ML worker here. People work in one of 3 ways: 1) Consumer Nvidia GPU cards on custom PCs 2) Self hosted shared server 3) Cloud infrastructure. There is no "GPU compute only card" that is widely used outside servers. > company PCs normally ban playing games on company PCs and the overlap of "needing max GPU compute" and "needing complicated 3D rendering tasks" is limited. The "don't play games thing" isn't a factor. Most companies just buy a 4090 or whatever, and if they have to tell staff not to play games, they say "don't play games". Fortnight runs just fine on pretty much anything anyway.
  - dathinab 4y ago
    my point is a card bought for doing GPU computer not being able to work as a normal graphic card is not a problem I'm aware that currently GPU compute only cards are not widly used outside of the server space. But that's not because people need the consumer GPU features (bedsides video decode) but because the economics of availability and cost lead to consumer GPUs being the best option (and this economic effects don't just apply to customers but also Nvidia itself).

Hooray_Darakian 4y ago

> AMD should invest into the full AI stack starting from training. https://www.amd.com/en/graphics/servers-solutions-rocm-ml https://www.amd.com/en/graphics/servers-solutions-rocm-ml > For this they need a product comparable to NVIDIA 4090, so that entry level researchers could use their hardware. Why is a high end product a requirement for entry level research?

jzwinck 4y ago
Because high end research uses a fleet of them, not just one.
kombine 4y ago
4090 (or 3090, 1080Ti and so on) is a high-end consumer GPU, but at the same time it is an entry level GPU for AI researchers. Don't forget that workstation cards (RTX 8000) let alone server-grade GPUs such as A100 are an order of magnitude more expensive.
- gymbeaux 4y ago
  I was doing ML stuff on a GTX 1060 a few years ago. As with everything, it depends on what you’re doing.
  - p1esk 4y ago
    Chances of you publishing something in ML improve proportionally to the amount of hardware you have access to. Or to put it another way, the less hardware you have, the smarter you have to be to publish something in ML.
- DeepYogurt 4y ago
  That does not sound like "entry level" research to me.
  - UncleEntity 4y ago
    IDK, I want to train up some AI and have the choice of using google colab or buying a GPU that can do it within a reasonable timeframe. Not going to be spending $10k like the tortoise-tts guy (was looking into that project last night) but $2k might be doable for a hobby project. Plus I’d have a computer at the end.
marcyb5st 4y ago
Because of VRAM. Even a simple model for language can easily max out a 4090. Also, ROC-M is a bit of a mess to setup. With Nvidia i just need to install cuda, cudnn and then pip install tensorflow/pytorch.

wmf 4y ago

MI200, MI250, and MI300 should work for training. They don't have an exact equivalent to the 4090 but that may be setting the bar too high. Nobody can deliver everything that Nvidia has but better.

kombine 4y ago
It will be harder for a small academic lab to afford MI300, whereas everyone can purchase a few cards costing $1500. And even if I had money to buy MI300, I wouldn't - it is a too risky investment, because we have no idea how suitable they are for common AI research workflows. They need to lower the entry bar, so that people can try out their hardware. Even 80% of the performance of 4090 would be enough at an appropriate price point.

modeless 4y ago

The hardware is not their major problem. They have been failing super hard at the software side of machine learning for a solid decade now. It seems like pure management incompetence to me. They need to invest a whole lot more in software, integrating their stuff directly into pytorch/TF/XLA/etc and making sure it works on consumer cards too. The investment would be paid back tenfold. The market is crying out for more competition for Nvidia and there's huge money to be made on the datacenter side but it all needs to work on the consumer side too.

hedgehog 4y ago
Last I checked they see deep learning training as a niche market, their strategy is to try to win big contracts (HPC etc) and then supply software specifically for that. Then "the community" will supply software. Having spent a bunch of time beating my head on this and related walls it's not clear to me that they're entirely wrong from an economic standpoint. Remember that 2/3 public cloud providers have their own chips as well as NVIDIA's so it would be tough to negotiate a good deal. As a user it's super irritating to be stuck on NVIDIA especially when Jensen gets up on stage to say "haha, Moore's law is over, stop expecting our products to get cheaper."
- marcyb5st 4y ago
  I hope they change their minds. At least now that generative models are becoming somewhat popular. I'd love to be able to get an AMD card to run generative models, but to the best of my knowledge, they only run on Nvidia hardware
  - nl 4y ago
    No personal experience, but you can actually get Stable Diffusion to run on AMD cards. It uses DirectML on Windows: https://gist.github.com/averad/256c507baa3dcc9464203dc14610d674 https://gist.github.com/averad/256c507baa3dcc9464203dc14610d... This is thanks to Microsoft, not AMD. On Linux you can use ROCm: https://www.videogames.ai/2022/11/06/Stable-Diffusion-AMD-GPU-ROCm-Linux.html https://www.videogames.ai/2022/11/06/Stable-Diffusion-AMD-GP... The horrible install processes and what a mess this is is all down to AMD.

hedgehog 4y ago

Big question is why? Would competing for entry level researchers buy them much?

deleted 4y ago
[deleted]
kombine 4y ago
Because entry level researchers shape the industry in the long term. I'm in academia, I worked at two universities and I have not seen a lab that uses non-NVIDIA hardware for research. Majority of graduates go to work in the industry.
- hedgehog 4y ago
  I used to believe this idea that the tools in academia would carry over to industry. Now I think it's only weakly true, that is there's not really a big barrier to switching to other options like TPU or maybe Trainium (haven't tried it). Supporting independent researchers gives you the Heroku problem, they may like your product but as they get more sophisticated and go to production they'll accept significant pain to save money, scale better, etc. You basically have to re-win that business from scratch and the technical tradeoffs are very different at that point.

dathinab 4y ago

> product comparable to NVIDIA 4090 no, they need a product good at training and gpu compute at a reasonable price that product doesn't need to be good at rendering, ray tracing and similar sure students and some independent contractors probably love getting both a good graphic card and a CUDA card in one and it makes it easier for people to experiment with it but company PCs normally ban playing games on company PCs and the overlap of "needing max GPU compute" and "needing complicated 3D rendering tasks" is limited. through having 1 product instead of two does make supply chain and pricing easier but then 4090 is by now in a price range where students are unlikely to afford it and people will think twice about buying it just to play around with GPU compute. So e.g. the 7900XTX having somewhat comparable GPU compute usability then a 4080 would have been good enough for the non company use case, where a dedicated compute-per-money cheaper GPU compute only card would be preferable for the company use case I think.

nl 4y ago
Long time ML worker here. People work in one of 3 ways: 1) Consumer Nvidia GPU cards on custom PCs 2) Self hosted shared server 3) Cloud infrastructure. There is no "GPU compute only card" that is widely used outside servers. > company PCs normally ban playing games on company PCs and the overlap of "needing max GPU compute" and "needing complicated 3D rendering tasks" is limited. The "don't play games thing" isn't a factor. Most companies just buy a 4090 or whatever, and if they have to tell staff not to play games, they say "don't play games". Fortnight runs just fine on pretty much anything anyway.
- dathinab 4y ago
  my point is a card bought for doing GPU computer not being able to work as a normal graphic card is not a problem I'm aware that currently GPU compute only cards are not widly used outside of the server space. But that's not because people need the consumer GPU features (bedsides video decode) but because the economics of availability and cost lead to consumer GPUs being the best option (and this economic effects don't just apply to customers but also Nvidia itself).