4 ms·

Given the current situation, this is wishful thinking/a long way off. Nvidia is considerably better at letting people get the job done on consumer grade GPUs th

Given the current situation, this is wishful thinking/a long way off. Nvidia is considerably better at letting people get the job done on consumer grade GPUs than AMD.

roenxi 3y ago
Given the amounts of money and attention being bought to bear? It'll be 12-24 months or less. Tonight I'm going to play around with LLMs. I'm probably going to go to bed early because my AMD graphics card running ROCm (on an unsupported system and a probably unsupported graphics card) will eventually cause a kernel panic. In some sense that is unforgivable. In other senses, they are only 1-2 bugs away from being perfectly good enough and competitive with Nvidia for me. That isn't much of a moat. It usually takes an hour or so for the drivers to collapse and for those hours everything works great.
- htrp 3y ago
  > It'll be 12-24 months or less. AMD also said this 24 months ago.
  - wmil 3y ago
    I think I've been hearing that ATI/AMD drivers are about to catch up with nVidia for 20 years.
    - avmich 3y ago
      You, for many definitions of "you", never work on a supercomputer. Despite the fact that today's smartphones are faster than a Cray-1 of the yesteryear. Similarly, you don't need to have ATI/AMD to catch the absolute performance crown from Nvidia, like AMD did several times with Intel, for you to use AMD solutions. We'll see how smart the market and the monopolist are this time around, yes, in the next 12-24 months... and maybe longer.
      - gyrovagueGeist 3y ago
        Well, most of the new DoE machines are MI300 based so AMD seems to be doing pretty well even at the supercomputer level :)
    - htrp 3y ago
      Good point. I forgot about the broken record that was gaming drivers. I was specifically referring to AMD's investments in a unified computing stack (their equivalent of CUDA) which were supposed to bring them to computational parity in the AI/ML.
- deleted 3y ago
  [deleted]
andrewstuart 3y ago
“AI Developers Are Buying Gaming GPUs in Bulk” https://www.extremetech.com/computing/here-we-go-again-ai-developers-are-buying-gaming-gpus-in-bulk https://www.extremetech.com/computing/here-we-go-again-ai-de... Retail gaming GPUs are available and they’re MUCH cheaper than cloud GPU. It’s happening now. AI software needs to make a choice too, does it only run in Nvidia, or does it make itself compatible with the mass market. For 50 years software developers have chosen to make their software work on cheap mass market devices.
- jcelerier 3y ago
  But retail gaming GPUs are overwhelmingly Nvidia? https://store.steampowered.com/hwsurvey/videocard/ https://store.steampowered.com/hwsurvey/videocard/
  - ElectricalUnion 3y ago
    Those overwhelmingly cheap Nvidia gaming GPUs got purchased for gaming use - and have usually low tensor compute capabilities and low VRAM ammounts so they're less desirable even for post-training quantization generative large language models.
  - andrewstuart 3y ago
    Nvidia imposes constraints on its drivers so you cannot really make full use of them and must go for high end data centre devices.
- michaelt 3y ago
  It's true that a lot of ML development has taken place on gaming GPUs, over the past decade and a half. And you can certainly get a long way with gaming GPUs in a lot of areas of ML - 8GB is enough to run things like stable diffusion, just about. However, some ML models (such as LLMs) demand a lot of vram to train. Simultaneously, a lot of gamers have been unimpressed by the latest generations of gaming GPUs due to the limited vram. nvidia's GTX-1070 released in 2016 with 8 GB of vram and an MSRP of $379. Then the RTX-2060 Super, with 8 GB of vram and an MSRP of $399. Then the RTX-3060 Ti, with 8 GB of vram and an MSRP of $399. Then the RTX-4060 Ti with - you guessed it - 8 GB of vram and an MSRP of $399. Some people think nvidia is deliberately being miserly with vram on gaming cards, to try and force ML users onto the $$$$$ data centre GPUs. I suspect this is what andrewstuart means by "Nvidia is pushing upmarket" - that they're deliberately letting their gaming products languish, in pursuit of the data centre market.
politelemon 3y ago
But surely it is in the best interest of the industry as a whole to ensure that there is parity across the competitors so they aren't milked forever? ie they could invest time and effort into making the AMD compatibility better, and the Intel Arc equivalent. I say this as a person who's diving right into the NVidia moat. "Well everyone gets NVIdia GPUs so that's what I should get".
- dragonwriter 3y ago
  > But surely it is in the best interest of the industry as a whole to ensure that there is parity across the competitors so they aren't milked forever? So what? “The industry as a whole” isn’t an actor that makes decisions.
- Certhas 3y ago
  Sure, this isn't in the hands of the industry as a whole, though. The question is: Does AMD think they can earn a better profit down the road than if they invest in/focus on something else (e.g. maintaining their lead over intel on x86, continue being dominant in the console gaming GPU sector, etc...).

Given the current situation, this is wishful thinking/a long way off. Nvidia is considerably better at letting people get the job done on consumer grade GPUs than AMD.

roenxi 3y ago

Given the amounts of money and attention being bought to bear? It'll be 12-24 months or less. Tonight I'm going to play around with LLMs. I'm probably going to go to bed early because my AMD graphics card running ROCm (on an unsupported system and a probably unsupported graphics card) will eventually cause a kernel panic. In some sense that is unforgivable. In other senses, they are only 1-2 bugs away from being perfectly good enough and competitive with Nvidia for me. That isn't much of a moat. It usually takes an hour or so for the drivers to collapse and for those hours everything works great.

htrp 3y ago
> It'll be 12-24 months or less. AMD also said this 24 months ago.
- wmil 3y ago
  I think I've been hearing that ATI/AMD drivers are about to catch up with nVidia for 20 years.
  - avmich 3y ago
    You, for many definitions of "you", never work on a supercomputer. Despite the fact that today's smartphones are faster than a Cray-1 of the yesteryear. Similarly, you don't need to have ATI/AMD to catch the absolute performance crown from Nvidia, like AMD did several times with Intel, for you to use AMD solutions. We'll see how smart the market and the monopolist are this time around, yes, in the next 12-24 months... and maybe longer.
    - gyrovagueGeist 3y ago
      Well, most of the new DoE machines are MI300 based so AMD seems to be doing pretty well even at the supercomputer level :)
  - htrp 3y ago
    Good point. I forgot about the broken record that was gaming drivers. I was specifically referring to AMD's investments in a unified computing stack (their equivalent of CUDA) which were supposed to bring them to computational parity in the AI/ML.
deleted 3y ago
[deleted]

andrewstuart 3y ago

“AI Developers Are Buying Gaming GPUs in Bulk” https://www.extremetech.com/computing/here-we-go-again-ai-developers-are-buying-gaming-gpus-in-bulk https://www.extremetech.com/computing/here-we-go-again-ai-de... Retail gaming GPUs are available and they’re MUCH cheaper than cloud GPU. It’s happening now. AI software needs to make a choice too, does it only run in Nvidia, or does it make itself compatible with the mass market. For 50 years software developers have chosen to make their software work on cheap mass market devices.

jcelerier 3y ago
But retail gaming GPUs are overwhelmingly Nvidia? https://store.steampowered.com/hwsurvey/videocard/ https://store.steampowered.com/hwsurvey/videocard/
- ElectricalUnion 3y ago
  Those overwhelmingly cheap Nvidia gaming GPUs got purchased for gaming use - and have usually low tensor compute capabilities and low VRAM ammounts so they're less desirable even for post-training quantization generative large language models.
- andrewstuart 3y ago
  Nvidia imposes constraints on its drivers so you cannot really make full use of them and must go for high end data centre devices.
michaelt 3y ago
It's true that a lot of ML development has taken place on gaming GPUs, over the past decade and a half. And you can certainly get a long way with gaming GPUs in a lot of areas of ML - 8GB is enough to run things like stable diffusion, just about. However, some ML models (such as LLMs) demand a lot of vram to train. Simultaneously, a lot of gamers have been unimpressed by the latest generations of gaming GPUs due to the limited vram. nvidia's GTX-1070 released in 2016 with 8 GB of vram and an MSRP of $379. Then the RTX-2060 Super, with 8 GB of vram and an MSRP of $399. Then the RTX-3060 Ti, with 8 GB of vram and an MSRP of $399. Then the RTX-4060 Ti with - you guessed it - 8 GB of vram and an MSRP of $399. Some people think nvidia is deliberately being miserly with vram on gaming cards, to try and force ML users onto the $$$$$ data centre GPUs. I suspect this is what andrewstuart means by "Nvidia is pushing upmarket" - that they're deliberately letting their gaming products languish, in pursuit of the data centre market.

politelemon 3y ago

But surely it is in the best interest of the industry as a whole to ensure that there is parity across the competitors so they aren't milked forever? ie they could invest time and effort into making the AMD compatibility better, and the Intel Arc equivalent. I say this as a person who's diving right into the NVidia moat. "Well everyone gets NVIdia GPUs so that's what I should get".

dragonwriter 3y ago
> But surely it is in the best interest of the industry as a whole to ensure that there is parity across the competitors so they aren't milked forever? So what? “The industry as a whole” isn’t an actor that makes decisions.
Certhas 3y ago
Sure, this isn't in the hands of the industry as a whole, though. The question is: Does AMD think they can earn a better profit down the road than if they invest in/focus on something else (e.g. maintaining their lead over intel on x86, continue being dominant in the console gaming GPU sector, etc...).