4 ms·

AMD is not too far behind. They can go wild in a few years.

behnamoh 3y ago
In the LLM world, each month is a year and each week is a month. AMD hasn't done anything substantive in the past two years since GPT-3/GPT-3.5. Almost every research paper implements their algorithm in CUDA. AMD can't beat software with good hardware.
- deleted 3y ago
  [deleted]
- brucethemoose2 3y ago
  There's more ROCm compatibility than you'd think. I'm at a startup, and we'd love to be using MI300Xs. Out stack works with it, they are amazing, our wallet is open... But we can't! We simply can't find any. It seems they are unobtanium for megacaps only.
- dartos 3y ago
  AMD has rocm and added a CUDA compat layer to it. Nvidia is in the limelight, but their product (GPU compute) is a commodity. Once someone else has it cheaper, then it’s a race to the bottom.
  - pests 3y ago
    They did not add a compat layer, where did you get this? The recent news was about AMD giving up on that path.
    - dartos 3y ago
      https://github.com/vosen/ZLUDA https://github.com/vosen/ZLUDA They still funded it and it was created.
      - pests 3y ago
        They only reason it is open source is because they decided not to go this direction. They gave it to us instead of tossing it into the trash. I don't know what you mean to imply by that.
        dartos 3y ago
        There’s nothing to imply. AMD funded a CUDA compatibility layer that now exists. Just because they don’t want it doesn’t mean it just vanishes or stops working.
- HWR_14 3y ago
  Didn't AMD just commit to a CUDA compatible api? Did I hallucinate that?
  - tomoyoirl 3y ago
    If you mean the one posted here earlier today, I believe that article was more like “we paid a contractor to implement this, and then decided not to use it, so per our terms it’s open source now.”
  - lhl 3y ago
    Well, you probably read a inaccurate headline about it. The project is called ZLUDA https://github.com/vosen/ZLUDA https://github.com/vosen/ZLUDA and it had a recent public update because of the opposite - AMD decide not to continue sponsoring work on it: > Shortly thereafter I got in contact with AMD and in early 2022 I have left Intel and signed a ZLUDA development contract with AMD. Once again I was asked for a far-reaching discretion: not to advertise the fact that AMD is evaluating ZLUDA and definitely not to make any commits to the public ZLUDA repo. After two years of development and some deliberation, AMD decided that there is no business case for running CUDA applications on AMD GPUs. > > One of the terms of my contract with AMD was that if AMD did not find it fit for further development, I could release it. Which brings us to today. It's worth noting that while ZLUDA is a very cool project, it's probably not so relevant for ML. Also from the README: > PyTorch received very little testing. ZLUDA's coverage of cuDNN APIs is very minimal (just enough to run ResNet-50) and realistically you won't get much running. > However if you are interested in trying it out you need to build it from sources with the settings below. Default PyTorch does not ship PTX and uses bundled NCCL which also builds without PTX: PyTorch has OOTB ROCm support btw and while there are some CUDA-only libraries I'd like (FA2 for RDNA, bitsandbytes, ctranslate2, FlashInfer among others), I think sponsoring direct porting/upstreaming compatibility of the libraries probably makes more sense. Also from the ZLUDA README: > ZLUDA offers limited support for performance libraries (cuDNN, cuBLAS, cuSPARSE, cuFFT, OptiX, NCCL).
  - zozbot234 3y ago
    Different kinds of compatibility. HIP is source compatible and officially supported. Zluda is the newly released project for running CUDA-compiled binaries.
dingi 3y ago
Oh please, AMD's offerings are worse than Nvidia's in every imaginable way except the open source nature of their Linux drivers. ROCm is shit. Nvidia actually supports CUDA on almost any Nvidia card. ROCm not so much.

In the LLM world, each month is a year and each week is a month. AMD hasn't done anything substantive in the past two years since GPT-3/GPT-3.5. Almost every research paper implements their algorithm in CUDA. AMD can't beat software with good hardware.

deleted 3y ago
[deleted]
brucethemoose2 3y ago
There's more ROCm compatibility than you'd think. I'm at a startup, and we'd love to be using MI300Xs. Out stack works with it, they are amazing, our wallet is open... But we can't! We simply can't find any. It seems they are unobtanium for megacaps only.
dartos 3y ago
AMD has rocm and added a CUDA compat layer to it. Nvidia is in the limelight, but their product (GPU compute) is a commodity. Once someone else has it cheaper, then it’s a race to the bottom.
- pests 3y ago
  They did not add a compat layer, where did you get this? The recent news was about AMD giving up on that path.
  - dartos 3y ago
    https://github.com/vosen/ZLUDA https://github.com/vosen/ZLUDA They still funded it and it was created.
    - pests 3y ago
      They only reason it is open source is because they decided not to go this direction. They gave it to us instead of tossing it into the trash. I don't know what you mean to imply by that.
      - dartos 3y ago
        There’s nothing to imply. AMD funded a CUDA compatibility layer that now exists. Just because they don’t want it doesn’t mean it just vanishes or stops working.
HWR_14 3y ago
Didn't AMD just commit to a CUDA compatible api? Did I hallucinate that?
- tomoyoirl 3y ago
  If you mean the one posted here earlier today, I believe that article was more like “we paid a contractor to implement this, and then decided not to use it, so per our terms it’s open source now.”
- lhl 3y ago
  Well, you probably read a inaccurate headline about it. The project is called ZLUDA https://github.com/vosen/ZLUDA https://github.com/vosen/ZLUDA and it had a recent public update because of the opposite - AMD decide not to continue sponsoring work on it: > Shortly thereafter I got in contact with AMD and in early 2022 I have left Intel and signed a ZLUDA development contract with AMD. Once again I was asked for a far-reaching discretion: not to advertise the fact that AMD is evaluating ZLUDA and definitely not to make any commits to the public ZLUDA repo. After two years of development and some deliberation, AMD decided that there is no business case for running CUDA applications on AMD GPUs. > > One of the terms of my contract with AMD was that if AMD did not find it fit for further development, I could release it. Which brings us to today. It's worth noting that while ZLUDA is a very cool project, it's probably not so relevant for ML. Also from the README: > PyTorch received very little testing. ZLUDA's coverage of cuDNN APIs is very minimal (just enough to run ResNet-50) and realistically you won't get much running. > However if you are interested in trying it out you need to build it from sources with the settings below. Default PyTorch does not ship PTX and uses bundled NCCL which also builds without PTX: PyTorch has OOTB ROCm support btw and while there are some CUDA-only libraries I'd like (FA2 for RDNA, bitsandbytes, ctranslate2, FlashInfer among others), I think sponsoring direct porting/upstreaming compatibility of the libraries probably makes more sense. Also from the ZLUDA README: > ZLUDA offers limited support for performance libraries (cuDNN, cuBLAS, cuSPARSE, cuFFT, OptiX, NCCL).
- zozbot234 3y ago
  Different kinds of compatibility. HIP is source compatible and officially supported. Zluda is the newly released project for running CUDA-compiled binaries.

dingi 3y ago

Oh please, AMD's offerings are worse than Nvidia's in every imaginable way except the open source nature of their Linux drivers. ROCm is shit. Nvidia actually supports CUDA on almost any Nvidia card. ROCm not so much.