4 ms·

You have no idea how deep the Nvidia lock-in goes. AMD only supports small fractions of the deep custom eco-systems built around models such as Stable Diffusio

You have no idea how deep the Nvidia lock-in goes. AMD only supports small fractions of the deep custom eco-systems built around models such as Stable Diffusion. Good luck getting half of the shit built for ComfyUI or Automatic1111 to run at all, and frequently those features are the killer ones which motivate folks to want to do AI in the first place. There's a reason why Nvidia is a first class citizen among all of the AI clouds. AMD is even behind Intel in this regard, because at least Intel had an existing robust ecosystem of accelerators (i.e. intel optimized sklearn/pytorch, OpenVino, etc) AMD can be 4x better in every way compared to Nvidia hardware wise but if they can't get developer communities to build around their equivalent to CUDA and its ecosystem than it's not looking rosy for AMDs future.

ClumsyPilot 3y ago
Which makes it even more stupid that AMD does not at least try to compensate for their shortcoming by selling large memory GPUs, there would be a reason to develop for team red
- brucethemoose2 3y ago
  Yeah. If there was a 40GB-48GB 7900 SKU, I would have bought it over a 3090 in a heartbeat, even at a big premium over the 24GB card. And I'd be debugging ROCm support in ML projects left and right. But I am not. The 24GB SKU is just not worth the trouble over a 3090, which is more RAM efficient out of the box. I can't pay $5K for a W7900.
latchkey 3y ago
> but if they can't get developer communities to build around their equivalent to CUDA and its ecosystem than it's not looking rosy for AMDs future. Dec 6th was an announcement from Lisa Su about the MI300x. It also included a full on commitment to ROCm as well as other projects like pyTorch. They are clearly aware of what you're talking about, and working on it. It isn't something that can magically be fixed over night, but it is something that people can complain about on HN over and over and over again...
- nemothekid 3y ago
  Some of us have been complaining for close to a decade and there's been multiple promises that have gone no where. It's better to keep complaining until the problem is actually fixed. CUDA is 16 years old and AMD has been "aware" of the issue for at least 10.
  - latchkey 3y ago
    How could they promise to fix something that didn't exist 10-16 years ago? In my eyes, the incentives weren't there before. But, they sure are now!
    - ihattendorf 3y ago
      And nvidia is killing it right now because they predicted the need _before_ the incentives were there.
      - latchkey 3y ago
        Nobody is arguing that AMD didn't get caught with their pants around their ankles or that they have a great reputation with developers. What matters is their statements and more importantly, actions, moving forward. Right now, they are cranking out ROCm releases on schedule, contributing to a bunch of AI open source projects, listening to crazy people like George Hotz and have released a GPU product that leapfrogs the H100 and is built on OAM-UBB, at 1/3 the FLOPS/$, while also promoting fast ethernet standards. I'd call that a pretty good start.
    - nemothekid 3y ago
      OpenCL did exist 10 years ago! Rocm has been around for 7 years. Pytorch is 7 years old and had CUDA support on day one and still doesn't have RocM support. Before the LLM craze people still wanted an alternative to nvidia and could see the writing on the wall. There was a bunch of complaining about Rocm and OpenCL didn't deliver. GeoHotz tried to get a RocM backend working and the examples just didn't work. You can't build a software platform by just willing it into existence. It means sitting down with developers, writing documentation and support users. It means tackling use cases other than llama so that users feel confident in building the next greatest thing on your platform. At it is today, AMD will always be playing catch up and people will continue to push nvidia because the latest and greatest will be on Cuda. I guess for those of spending money on real gpus for the last 10 years, nvidia was incentivized to take our money but amd wasn't.
brucethemoose2 3y ago
This is a pet peeve of mine, but A1111 is not a great example of a ML project. The backend codebase is a mess compared to diffusers, its slow even on Nvidia, its history is filled with controversy (including license breaking and outright piracy in the codebase), the frontend is buggy and unstable. I appreciate how it was a pioneer in 2022-H1 2023, but I am somewhat annoyed its still the de facto SD backend. Better maintained inference projects are picking up considerable AMD/Intel support. Sometimes even Apple Silicon support.
- Der_Einzige 3y ago
  The only real competitor is ComfyUI, because without the huge ecosystem around your other front end, you lose MASSIVE capabilities, for example, losing this is a big mistake: https://github.com/ljleb/prompt-fusion-extension https://github.com/ljleb/prompt-fusion-extension
  - jadbox 3y ago
    Is there something like that extension for comfy yet?
- pseudo0 3y ago
  At least on Nvidia, A1111 has had near identical performance to ComfyUI on SDXL and SD1.5 for me. Besides that, there were typical minor problems that happen when some guy's hobby project goes viral and suddenly is under massive scrutiny. It was the first "good enough" interface and now there is massive community inertia with extensions, plugins, etc. ComfyUI has found a place in the image pipeline design niche, but A1111 still seems like the best choice for new users who just want to spin up Stable Diffusion and generate some art for a D&D campaign or whatever.
  - brucethemoose2 3y ago
    Thats because ComfyUI is also based on the old SAI backend, not diffusers like, say, VoltaML.

ClumsyPilot 3y ago

Which makes it even more stupid that AMD does not at least try to compensate for their shortcoming by selling large memory GPUs, there would be a reason to develop for team red

brucethemoose2 3y ago
Yeah. If there was a 40GB-48GB 7900 SKU, I would have bought it over a 3090 in a heartbeat, even at a big premium over the 24GB card. And I'd be debugging ROCm support in ML projects left and right. But I am not. The 24GB SKU is just not worth the trouble over a 3090, which is more RAM efficient out of the box. I can't pay $5K for a W7900.

latchkey 3y ago

> but if they can't get developer communities to build around their equivalent to CUDA and its ecosystem than it's not looking rosy for AMDs future. Dec 6th was an announcement from Lisa Su about the MI300x. It also included a full on commitment to ROCm as well as other projects like pyTorch. They are clearly aware of what you're talking about, and working on it. It isn't something that can magically be fixed over night, but it is something that people can complain about on HN over and over and over again...

nemothekid 3y ago
Some of us have been complaining for close to a decade and there's been multiple promises that have gone no where. It's better to keep complaining until the problem is actually fixed. CUDA is 16 years old and AMD has been "aware" of the issue for at least 10.
- latchkey 3y ago
  How could they promise to fix something that didn't exist 10-16 years ago? In my eyes, the incentives weren't there before. But, they sure are now!
  - ihattendorf 3y ago
    And nvidia is killing it right now because they predicted the need _before_ the incentives were there.
    - latchkey 3y ago
      Nobody is arguing that AMD didn't get caught with their pants around their ankles or that they have a great reputation with developers. What matters is their statements and more importantly, actions, moving forward. Right now, they are cranking out ROCm releases on schedule, contributing to a bunch of AI open source projects, listening to crazy people like George Hotz and have released a GPU product that leapfrogs the H100 and is built on OAM-UBB, at 1/3 the FLOPS/$, while also promoting fast ethernet standards. I'd call that a pretty good start.
  - nemothekid 3y ago
    OpenCL did exist 10 years ago! Rocm has been around for 7 years. Pytorch is 7 years old and had CUDA support on day one and still doesn't have RocM support. Before the LLM craze people still wanted an alternative to nvidia and could see the writing on the wall. There was a bunch of complaining about Rocm and OpenCL didn't deliver. GeoHotz tried to get a RocM backend working and the examples just didn't work. You can't build a software platform by just willing it into existence. It means sitting down with developers, writing documentation and support users. It means tackling use cases other than llama so that users feel confident in building the next greatest thing on your platform. At it is today, AMD will always be playing catch up and people will continue to push nvidia because the latest and greatest will be on Cuda. I guess for those of spending money on real gpus for the last 10 years, nvidia was incentivized to take our money but amd wasn't.

brucethemoose2 3y ago

This is a pet peeve of mine, but A1111 is not a great example of a ML project. The backend codebase is a mess compared to diffusers, its slow even on Nvidia, its history is filled with controversy (including license breaking and outright piracy in the codebase), the frontend is buggy and unstable. I appreciate how it was a pioneer in 2022-H1 2023, but I am somewhat annoyed its still the de facto SD backend. Better maintained inference projects are picking up considerable AMD/Intel support. Sometimes even Apple Silicon support.

Der_Einzige 3y ago
The only real competitor is ComfyUI, because without the huge ecosystem around your other front end, you lose MASSIVE capabilities, for example, losing this is a big mistake: https://github.com/ljleb/prompt-fusion-extension https://github.com/ljleb/prompt-fusion-extension
- jadbox 3y ago
  Is there something like that extension for comfy yet?
pseudo0 3y ago
At least on Nvidia, A1111 has had near identical performance to ComfyUI on SDXL and SD1.5 for me. Besides that, there were typical minor problems that happen when some guy's hobby project goes viral and suddenly is under massive scrutiny. It was the first "good enough" interface and now there is massive community inertia with extensions, plugins, etc. ComfyUI has found a place in the image pipeline design niche, but A1111 still seems like the best choice for new users who just want to spin up Stable Diffusion and generate some art for a D&D campaign or whatever.
- brucethemoose2 3y ago
  Thats because ComfyUI is also based on the old SAI backend, not diffusers like, say, VoltaML.