4 ms·
The article highlights that onnx, deepspeed, pytorch and tensorflow all support these AMD processors
by kibibu 3y ago
The article highlights that onnx, deepspeed, pytorch and tensorflow all support these AMD processors
- CharlesW 3y agoThanks, so I think you're saying that there are two related things happening that are eliminating NVIDIA's CUDA moat: (1) Developers are mostly using libraries instead of writing "to the metal" (CUDA), and (2) many popular libraries have added support for AMD GPUs. Does that capture it?
- latchkey 3y agohttps://www.databricks.com/blog/training-llms-scale-amd-mi250-gpus https://www.databricks.com/blog/training-llms-scale-amd-mi25... https://embeddedllm.com/blog/vllm_rocm/ https://embeddedllm.com/blog/vllm_rocm/
- brucethemoose2 3y agoIt's the opposite in my experience. Pure PyTorch mostly works OK, but some libraries implementing crazy optimized, hand written kernels and such will have some trouble. So (for instance) maybe you can run an LLM in a particular PyTorch framework, but flash attention 2 doesn't support your AMD card, so performance and memory use takes a hit. Or maybe the library works on an Intel XPU with like 5 changed lines in the entire library (rename "cuda" to "xpu"), but no one bothered to add it, or maybe the dev doesn't even want to support the PR.
- sevagh 3y agoNot that I maintain any hugely popular deep learning code, but my own "hobby"/dev hesitance at adding multiple devices is that I don't own them to test them. If I have an NVIDIA GPU, the only PyTorch backend I can promise even "somewhat works on my machine" is CUDA.
- tails4e 3y agoAMD also have HIP which is almost exactly CUDA, but target agnostic, so basically now CUDA code will now run on AMD hardware. So they have the low level via HIP, and have also integrated into the libraries like pytorch. It's a great bit chip out of that moat.