5 ms·
AMD GPUs are still mainly used only for inference. Anyone who wants to use CUDA continues to be forced to buy Nvidia hardware.
by metadat 2y ago
AMD GPUs are still mainly used only for inference.
Anyone who wants to use CUDA continues to be forced to buy Nvidia hardware.
- dleink 2y agoHow far along are efforts to run CUDA on other hardware?
- qeternity 2y agoInference is the main cost involved in deploying at scale…
- bravetraveler 2y agoGPUs are only part of the stack, too. Be careful building hegemony. Don't forget the network.
- JonChesterfield 2y agoA few years ago everything had to be CUDA. Opencl and so forth were unacceptable. Now people say CUDA is better optimised than ROCm. Fewer claims that ROCm never works, more that it leaves performance on the table. You've just offered that people are using AMDGPU in production for inference. I think some are using it for training. That software moat is looking rather empty to me. And good riddance, the cuda programming model is nasty. The MI300A is gorgeous. Massive APU thing. Single block of fast memory that x64 threads and gpu kernels have atomic RMW access to. It is the evolution beyond the separate accelerator concept and the whole language patterns around "offloading" kernels is kind of nonsense on it. The gpu and x64 cores are peers. Viable alternatives to cuda are coming online at the same time as the hardware moves beyond it. It's a liability, not a moat.
- deleted 2y ago[deleted]