42 ms·

But without CUDA it’s gotta be DOA no? CUDA is what makes Nvidia’s hardware valuable. AMD should finance a competitor. They’ve had strong hardware but it’s the

But without CUDA it’s gotta be DOA no? CUDA is what makes Nvidia’s hardware valuable. AMD should finance a competitor. They’ve had strong hardware but it’s the software that’s hurting them from what I’ve seen.

wmf 3y ago
AMD is now starting to have decent software support.
- JonChesterfield 3y ago
  It's somewhat sad that AMD bet on open source over proprietary and then approximately zero people stepped up to help. Easier to use cuda and complain I suppose. Fortunately they've stuck with the open source plan anyway and just written it themselves.
  - wmf 3y ago
    Open source people would rather spend their time reverse engineering Nvidia, Mali, and Apple GPUs than helping AMD.
    - striking 3y ago
      Look, I love AMD, but there are different groups of open source people here. One group is trying to broaden the support for desktop Linux across all hardware, and the other group is trying to crunch numbers with GPUs and build support for that. There isn't a super strong overlap, and I think the former group is currently larger than the latter.
    - ahartmetz 3y ago
      AMD's paid linux driver team is moving relatively quickly using some documentation and tools (simulators) under NDA. It is difficult to contribute usefully to the core of that effort without getting hired by AMD. AFAIK, that is why not many hobbyists are contributing.
      - JonChesterfield 3y ago
        I would guess the documentation is something of a hindrance. I'm in compilers rather than drivers, where the public docs are essentially the pdfs at https://gpuopen.com/amd-isa-documentation/ https://gpuopen.com/amd-isa-documentation/. It's a lot better than the nvidia equivalent but it would also be rough going implementing a compiler based on those alone; too many instructions with names but unspecified semantics. The driver / firmware story is probably similar.
        ahartmetz 3y ago
        Oh, I thought I was missing something, maybe another document with more explanations. Thanks for clearing that up. I was actually motivated to contribute to the AMD drivers at some point and did land a couple of patches, but wasn't looking to switch career paths. The drivers have become quite good without me anyway, so no regrets :)
    - kcb 3y ago
      The highest performing open source graphics driver available anywhere is RADV.
deaddodo 3y ago
They have a competitor, it's just less popular than CUDA: https://www.amd.com/en/products/software/rocm.html https://www.amd.com/en/products/software/rocm.html Before that, they were pushing heavily for standard OpenCL, but that failed because the hardware wasn't as competitive and the ecosystem/tooling barren.
- atemerev 3y ago
  ROCm is not just unpopular, it is so boilerplate-ridden, it is nearly unusable. It takes about 5x more code to do things there compared to CUDA. Yes, “just use libraries”. But as Andrej Karpathy used to say, “I don't need some library holding my hand and providing abstractions. Real men command GPUs with their own raw kernel code”. (And, incidentally, there are much less libraries for ROCm, for this reason. Somebody should write them, and why do that with something that takes 5x more code to do the same thing?)
  - kcb 3y ago
    Why? I understand the framework setup may not be as polished as CUDA, but I was under the impression HIP is the primary kernel language supported by ROCm and at first glance it's basically a 1:1 CUDA clone.
    - atemerev 3y ago
      Hm, apparently, in the last years, HIP became much integrated into ROCm as the principal way of doing things. I was referring to the situation as of 2-3 years ago, when plain ROCm was a chore to write. OK, I retract my statement.
  - JonChesterfield 3y ago
    Where do you get the 5x code idea from? Hip and cuda are almost regex-equivalent, openmp is the same language on each.
    - atemerev 3y ago
screye 3y ago
Rocm will only get adoption when Cuda code can be ported over with 1 click. At the very least, pytorch has to work effortlessly across GPUs, down to the less popular and odd workarounds people build into their pyotrch code. Even if RocM reaches parity, the first mover advantage for Cuda is too large. RocM porting has to be literally effortless. Everything else is DOA.
- screye 3y ago
  Edit- That being said, if you a cloud provider and need to scale up a bunch of basically similar transformers models, then it should be an easy sell for AMD.
- wmf 3y ago
  The facts don't bear this out. Various companies have already ported to AMD even though it isn't effortless.
- com2kid 3y ago
  Counterpoint, at scale a month of training a GPU cost more than an engineer now days. And the large companies (Meta, Microsoft) buy these GPUs by the thousands upon thousands. Having a small team of engineers spend a couple months porting code over is well worth it for even modest cost reductions. These may not be useful to smaller companies working in the AI space, but odds are to sellout, all AMD really needs are the sales contracts that they've already announced.
- Der_Einzige 3y ago
  People here don't want to hear the truth, but this is the truth.
trynumber9 3y ago
Depends on your definition of DOA. For the big customers who can't get as many H100 as they want it'll be an option. They have the resources to make sure their workload does in fact work and then scale out hardware appropriately. And it might not even be a downgrade: MI300X has more bandwidth and larger shared memory space. And there's MI300A which has a combined CPU+GPU with shared memory space which already has supercomputer customers. You can say it's DOA relative to Hopper but MI300 will be AMD's most successful GPGPU yet.
bratao 3y ago
I don´t think so. There are few algorithms that cover the majority of use. For example in NLP a fast LLM (such as LLAMA) training/prediction and support for Hugging face transformers (BERT and such) would cover 95%+ of NLP use-cases that need a GPU.
kemotep 3y ago
> AMD should finance a competitor. If they can’t afford to out R&D Nvidia today, how could they afford to fund a competitor? In the past 20 years the only serious new entrant to the discrete GPU market has been Intel and they are also far behind Nvidia and CUDA.
lanza 3y ago
PyTorch and Triton don't use CUDA. An extraordinary volume of big tech compute only cares about the PTX “isa” for NVIDIA.

wmf 3y ago

AMD is now starting to have decent software support.

JonChesterfield 3y ago
It's somewhat sad that AMD bet on open source over proprietary and then approximately zero people stepped up to help. Easier to use cuda and complain I suppose. Fortunately they've stuck with the open source plan anyway and just written it themselves.
- wmf 3y ago
  Open source people would rather spend their time reverse engineering Nvidia, Mali, and Apple GPUs than helping AMD.
  - striking 3y ago
    Look, I love AMD, but there are different groups of open source people here. One group is trying to broaden the support for desktop Linux across all hardware, and the other group is trying to crunch numbers with GPUs and build support for that. There isn't a super strong overlap, and I think the former group is currently larger than the latter.
  - ahartmetz 3y ago
    AMD's paid linux driver team is moving relatively quickly using some documentation and tools (simulators) under NDA. It is difficult to contribute usefully to the core of that effort without getting hired by AMD. AFAIK, that is why not many hobbyists are contributing.
    - JonChesterfield 3y ago
      I would guess the documentation is something of a hindrance. I'm in compilers rather than drivers, where the public docs are essentially the pdfs at https://gpuopen.com/amd-isa-documentation/ https://gpuopen.com/amd-isa-documentation/. It's a lot better than the nvidia equivalent but it would also be rough going implementing a compiler based on those alone; too many instructions with names but unspecified semantics. The driver / firmware story is probably similar.
      - ahartmetz 3y ago
        Oh, I thought I was missing something, maybe another document with more explanations. Thanks for clearing that up. I was actually motivated to contribute to the AMD drivers at some point and did land a couple of patches, but wasn't looking to switch career paths. The drivers have become quite good without me anyway, so no regrets :)
  - kcb 3y ago
    The highest performing open source graphics driver available anywhere is RADV.

deaddodo 3y ago

They have a competitor, it's just less popular than CUDA: https://www.amd.com/en/products/software/rocm.html https://www.amd.com/en/products/software/rocm.html Before that, they were pushing heavily for standard OpenCL, but that failed because the hardware wasn't as competitive and the ecosystem/tooling barren.

atemerev 3y ago
ROCm is not just unpopular, it is so boilerplate-ridden, it is nearly unusable. It takes about 5x more code to do things there compared to CUDA. Yes, “just use libraries”. But as Andrej Karpathy used to say, “I don't need some library holding my hand and providing abstractions. Real men command GPUs with their own raw kernel code”. (And, incidentally, there are much less libraries for ROCm, for this reason. Somebody should write them, and why do that with something that takes 5x more code to do the same thing?)
- kcb 3y ago
  Why? I understand the framework setup may not be as polished as CUDA, but I was under the impression HIP is the primary kernel language supported by ROCm and at first glance it's basically a 1:1 CUDA clone.
  - atemerev 3y ago
    Hm, apparently, in the last years, HIP became much integrated into ROCm as the principal way of doing things. I was referring to the situation as of 2-3 years ago, when plain ROCm was a chore to write. OK, I retract my statement.
- JonChesterfield 3y ago
  Where do you get the 5x code idea from? Hip and cuda are almost regex-equivalent, openmp is the same language on each.
  - atemerev 3y ago

screye 3y ago

Rocm will only get adoption when Cuda code can be ported over with 1 click. At the very least, pytorch has to work effortlessly across GPUs, down to the less popular and odd workarounds people build into their pyotrch code. Even if RocM reaches parity, the first mover advantage for Cuda is too large. RocM porting has to be literally effortless. Everything else is DOA.

screye 3y ago
Edit- That being said, if you a cloud provider and need to scale up a bunch of basically similar transformers models, then it should be an easy sell for AMD.
wmf 3y ago
The facts don't bear this out. Various companies have already ported to AMD even though it isn't effortless.
com2kid 3y ago
Counterpoint, at scale a month of training a GPU cost more than an engineer now days. And the large companies (Meta, Microsoft) buy these GPUs by the thousands upon thousands. Having a small team of engineers spend a couple months porting code over is well worth it for even modest cost reductions. These may not be useful to smaller companies working in the AI space, but odds are to sellout, all AMD really needs are the sales contracts that they've already announced.
Der_Einzige 3y ago
People here don't want to hear the truth, but this is the truth.

trynumber9 3y ago

Depends on your definition of DOA. For the big customers who can't get as many H100 as they want it'll be an option. They have the resources to make sure their workload does in fact work and then scale out hardware appropriately. And it might not even be a downgrade: MI300X has more bandwidth and larger shared memory space. And there's MI300A which has a combined CPU+GPU with shared memory space which already has supercomputer customers. You can say it's DOA relative to Hopper but MI300 will be AMD's most successful GPGPU yet.

bratao 3y ago

I don´t think so. There are few algorithms that cover the majority of use. For example in NLP a fast LLM (such as LLAMA) training/prediction and support for Hugging face transformers (BERT and such) would cover 95%+ of NLP use-cases that need a GPU.

kemotep 3y ago

> AMD should finance a competitor. If they can’t afford to out R&D Nvidia today, how could they afford to fund a competitor? In the past 20 years the only serious new entrant to the discrete GPU market has been Intel and they are also far behind Nvidia and CUDA.

lanza 3y ago

PyTorch and Triton don't use CUDA. An extraordinary volume of big tech compute only cares about the PTX “isa” for NVIDIA.