5 ms·

I'm still blown away that AMD hasn't made it their top priority. I've said this for years. If I was AMD I would spend billions upon billions if necessary to mak

I'm still blown away that AMD hasn't made it their top priority. I've said this for years. If I was AMD I would spend billions upon billions if necessary to make a CUDA compatibility layer for AMD. It would certainly still pay off, and it almost certainly wouldn't cost that much.

andy_ppp 8mo ago
Moving target, honestly just get PyTorch working fully (loads of stuff just doesn’t work on AMD hardware) and also make it work on all graphics cards from a certain generation. The matrix of support needed GFX cards, architectures and software together is quite astounding but still yes that should have at least that working and equivalent custom kernels.
- colordrops 8mo ago
  That would be a great start.
- spacebanana7 8mo ago
  The headline that PyTorch has full compatibility on all AMD GPUs would increase their stock by > $50 billion overnight. They should do it even if it takes 500 engineers and 2 years.
  - DonThomasitos 8mo ago
    Does anybody really understand why this hasn‘t been done? I know about ongoing efforts but is it really THAT difficult?
    - andy_ppp 8mo ago
      You know it’s probably a combination of things but mostly that AMD do not have a capable software team… probably not the individuals but the managers likely don’t have a clue.
      - touisteur 8mo ago
        Aaaaaaand torch is not a simple easy target. You don't just want support but high-performance optimized support on a pretty-complex moving target... maybe better/easier than CUDA but not that much it seems.
ddtaylor 8mo ago
AMD did hire someone to do this and IIRC he did, but they were afraid of Nvidia lawyers and he released it outside of the company?
- colordrops 8mo ago
  Surely they could hire some good lawyers if that means they make billions upon billions? AFAIK there's nothing illegal about creating compatibility layers. Otherwise WINE would have shut down long ago.
  - M95D 8mo ago
    Depends on what code they wrote. If they used LLMs to write it, it could contain proprietary nvidia parts. Someone would then have to review that, but can't, because maybe the nvidia code that came from the LLM isn't even public. So the strategy to publish independently, wait and see if nvidia lawyers have anything to say about it, would be a very smart move.
    - ddtaylor 8mo ago
      The Oracle case was about the stubs of the API being considered copyrighted I believe. The argument wasn't that Google used any of their code, it was that by using the same functional names they were making a thought crime.
  - zvr 8mo ago
    The last time something similar happened (Google vs Oracle), the legal battles lasted more than a decade. It would be a very bold decision by ARM to commit to this strategy (implement "CUDA" and fight it out in courts).
    - gzread 8mo ago
      That means Google got to use Java for a decade. A decade-long legal battle is great news for whoever seems to be in the wrong, as long as they can still afford lawyers. Remember, they don't claw back dividends or anything.
woctordho 8mo ago
They've been doing it all the time and it's called HIP. Nowadays it works pretty well on a few supported GPUs (CDNA 3 and RDNA 4).
- mathisfun123 8mo ago
  it's astounding to me how many people pop off about "AMD SHOULD SUPPORT CUDA" not knowing that HIP (and hipify) has been around for literally a decade now.
  - colordrops 8mo ago
    Please explain to me why all the major players are buying Nvidia then? Is HIP a drop in replacement? No. You have to port every piece of software you want to use. It's ridiculous to call this a solution.
    - woctordho 8mo ago
      Major players in China don't play like that. MooreThreads, Lisuan, and many other smaller companies all have their own porting kits, which are basically copied from HIP. They just port every piece of software and it just works. If you want to fight against Nvidia monopoly, then don't just rant, but buy a GPU other than Nvidia and build on it. Check my GitHub and you'll see what I'm doing.
    - mathisfun123 8mo ago
      > Is HIP a drop in replacement? No. You don't understand what HIP is - HIP is AMD's runtime API. it resembles CUDA runtime APIs but it's not the same thing and it doesn't need to be - the hard part of porting CUDA isn't the runtime APIs. hipify is the thing that translates both runtime and kernels. Now is hipify a drop-in replacement? No of course but because the two vendors have different architectures. So it's absolutely laughable to imagine that some random could come anywhere near "drop-in replacement" when AMD can't (again: because of fundamental architecture differences).
      - 8mo ago

andy_ppp 8mo ago

Moving target, honestly just get PyTorch working fully (loads of stuff just doesn’t work on AMD hardware) and also make it work on all graphics cards from a certain generation. The matrix of support needed GFX cards, architectures and software together is quite astounding but still yes that should have at least that working and equivalent custom kernels.

colordrops 8mo ago
That would be a great start.
spacebanana7 8mo ago
The headline that PyTorch has full compatibility on all AMD GPUs would increase their stock by > $50 billion overnight. They should do it even if it takes 500 engineers and 2 years.
- DonThomasitos 8mo ago
  Does anybody really understand why this hasn‘t been done? I know about ongoing efforts but is it really THAT difficult?
  - andy_ppp 8mo ago
    You know it’s probably a combination of things but mostly that AMD do not have a capable software team… probably not the individuals but the managers likely don’t have a clue.
    - touisteur 8mo ago
      Aaaaaaand torch is not a simple easy target. You don't just want support but high-performance optimized support on a pretty-complex moving target... maybe better/easier than CUDA but not that much it seems.

ddtaylor 8mo ago

AMD did hire someone to do this and IIRC he did, but they were afraid of Nvidia lawyers and he released it outside of the company?

colordrops 8mo ago
Surely they could hire some good lawyers if that means they make billions upon billions? AFAIK there's nothing illegal about creating compatibility layers. Otherwise WINE would have shut down long ago.
- M95D 8mo ago
  Depends on what code they wrote. If they used LLMs to write it, it could contain proprietary nvidia parts. Someone would then have to review that, but can't, because maybe the nvidia code that came from the LLM isn't even public. So the strategy to publish independently, wait and see if nvidia lawyers have anything to say about it, would be a very smart move.
  - ddtaylor 8mo ago
    The Oracle case was about the stubs of the API being considered copyrighted I believe. The argument wasn't that Google used any of their code, it was that by using the same functional names they were making a thought crime.
- zvr 8mo ago
  The last time something similar happened (Google vs Oracle), the legal battles lasted more than a decade. It would be a very bold decision by ARM to commit to this strategy (implement "CUDA" and fight it out in courts).
  - gzread 8mo ago
    That means Google got to use Java for a decade. A decade-long legal battle is great news for whoever seems to be in the wrong, as long as they can still afford lawyers. Remember, they don't claw back dividends or anything.

woctordho 8mo ago

They've been doing it all the time and it's called HIP. Nowadays it works pretty well on a few supported GPUs (CDNA 3 and RDNA 4).

mathisfun123 8mo ago
it's astounding to me how many people pop off about "AMD SHOULD SUPPORT CUDA" not knowing that HIP (and hipify) has been around for literally a decade now.
- colordrops 8mo ago
  Please explain to me why all the major players are buying Nvidia then? Is HIP a drop in replacement? No. You have to port every piece of software you want to use. It's ridiculous to call this a solution.
  - woctordho 8mo ago
    Major players in China don't play like that. MooreThreads, Lisuan, and many other smaller companies all have their own porting kits, which are basically copied from HIP. They just port every piece of software and it just works. If you want to fight against Nvidia monopoly, then don't just rant, but buy a GPU other than Nvidia and build on it. Check my GitHub and you'll see what I'm doing.
  - mathisfun123 8mo ago
    > Is HIP a drop in replacement? No. You don't understand what HIP is - HIP is AMD's runtime API. it resembles CUDA runtime APIs but it's not the same thing and it doesn't need to be - the hard part of porting CUDA isn't the runtime APIs. hipify is the thing that translates both runtime and kernels. Now is hipify a drop-in replacement? No of course but because the two vendors have different architectures. So it's absolutely laughable to imagine that some random could come anywhere near "drop-in replacement" when AMD can't (again: because of fundamental architecture differences).
    - 8mo ago