2 ms·

I’m asking at a lower level than this, CUDA presumably has a list of functionality for GPGPU stuff like tensors, loading data, splitting up training, and buildi

I’m asking at a lower level than this, CUDA presumably has a list of functionality for GPGPU stuff like tensors, loading data, splitting up training, and building pipelines of networks /attention stuff that can efficiently fit neural networks to many sorts of data. Why is it so difficult for other manufacturers to provide a compatible layer? If Apple can make Direct X 12 work on Apple Silicon surely AMD should be able to make CUDA (which has to be much simpler that DX12) work on their graphics cards? Is there some fundamental architectural differences that stop this from working?

empyrrhicist 3y ago
There might be intellectual property concerns with "directly" implementing CUDA, and the architectures are (as I understand it) a bit different. That doesn't explain why they don't support something with similar broad compatibility though, as the actual card capabilities are very similar.
hedgehog 3y ago
There's nothing conceptually hard but it's really a lot of work. In addition to the items you listed there's the actual compute kernels or compiler to generate those, and then porting frameworks over (PyTorch etc), and then doing the level of testing, documentation, and ongoing maintenance to make an alternative platform a reasonable idea for end users. The pitch for buying NVIDIA hardware is that existing tools, example code, and third party research will more or less work and perform well out of the box. Edit: Going back to your original question, the main thing that makes CUDA so special is NVIDIA has already poured billions of dollars into all of this infrastructure and credibly will keep doing so.
mrguyorama 3y ago
Sure, AMD could write a CUDA emulator (if it was legal) for AMD GPUs, but if it's one tenth the performance, whats the point?
- empyrrhicist 3y ago
  There's no real reason it would need to be 1/10 the performance though, depending on the kernel.
- JonChesterfield 3y ago
  Compiler. It would look a lot like HIP and run at about the same performance that a cuda implementation would.

empyrrhicist 3y ago

There might be intellectual property concerns with "directly" implementing CUDA, and the architectures are (as I understand it) a bit different. That doesn't explain why they don't support something with similar broad compatibility though, as the actual card capabilities are very similar.

hedgehog 3y ago

There's nothing conceptually hard but it's really a lot of work. In addition to the items you listed there's the actual compute kernels or compiler to generate those, and then porting frameworks over (PyTorch etc), and then doing the level of testing, documentation, and ongoing maintenance to make an alternative platform a reasonable idea for end users. The pitch for buying NVIDIA hardware is that existing tools, example code, and third party research will more or less work and perform well out of the box. Edit: Going back to your original question, the main thing that makes CUDA so special is NVIDIA has already poured billions of dollars into all of this infrastructure and credibly will keep doing so.

mrguyorama 3y ago

Sure, AMD could write a CUDA emulator (if it was legal) for AMD GPUs, but if it's one tenth the performance, whats the point?

empyrrhicist 3y ago
There's no real reason it would need to be 1/10 the performance though, depending on the kernel.
JonChesterfield 3y ago
Compiler. It would look a lot like HIP and run at about the same performance that a cuda implementation would.