3 ms·

People don't seem particularly motivated to move away from CUDA to me. I've been poking around various models and tooling over the last couple months, and they

by bragr 3y ago

People don't seem particularly motivated to move away from CUDA to me. I've been poking around various models and tooling over the last couple months, and they pretty much all have something like device = "cuda" if torch.cuda.is_available() else "cpu" and I've yet to see a single one implement the AMD NN middleware: https://www.amd.com/en/developer/zendnn.html https://www.amd.com/en/developer/zendnn.html

PeterisP 3y ago
This is a bit of a chicken and egg problem, because the developers of these projects probably don't even have relevant AMD devices to test that, because why would they? The only reasonable way to break the cycle would be proactive intervention by AMD to contribute code and testing and easy "works out of the box" installation to all the major popular projects to add AMD support so they can sell more their hardware later, but I'm not seeing AMD doing that.
- lainga 3y ago
  Maybe the AMD device will just have to announce it's (KHTML, like Gecko).
- mschuetz 3y ago
  CUDA is just way to good in terms of UX with nothing platform-independent coming close to it. I wish there was a competitor, but there simply isn't. I wish AMD&Intel would extend compute shaders with pointers&pointer casting, arbitrary large buffers instead of just 4GB, device-wide sync, and function pointers. Those are kinda my must-have functionality. Even better, just use C++ for compute shaders.
  - JonChesterfield 3y ago
    Freestanding C++ can be compiled to amdgpu or nvptx using clang. There are rough edges but it's usable.
- JonChesterfield 3y ago
  Like the pytorch one? Or the onnx? Or Triton? There's loads of these machine learning things, each of which built their thing assuming Nvidia, to which AMD is stoically working to implement support for their hardware. It's a task of unbounded scope with unclear return on investment and they're doing it anyway.
latchkey 3y ago
You didn't watch the announcement on the MI300x. You don't have to change that line to use AMD.
kaelinl 3y ago
I haven't used it personally, but my understanding is that AMD/ROCm-accelerated backends for PyTorch overload the "cuda" device and module so that ROCm shows up as CUDA for feature testing. They want to make a transition seamless, and lots of existing code checks for CUDA, so they do what's necessary to make that existing code run.
- bragr 3y ago
  Which tools actually pull down those packages though? If anything that's a counter point to the Intel CEO: industry is adopting CUDA as the defacto standard.
- mappu 3y ago
  That seems to be the case for me out-of-the-box. $ python3 -m venv . $ ./bin/pip3 install torch --index-url https://download.pytorch.org/whl/rocm5.6 $ ./bin/python3 -c 'import torch; print([torch.cuda.is_available(), torch.cuda.get_device_name(0)])' [True, 'AMD Radeon RX 6600 XT']
muxr 3y ago
zendnn is for CPUs. Pytorch ROCm version doesn't require you to change a single line of code. So all the same semantics like `device = "cuda"` work the same when using AMD GPUs.

by bragr 3y ago

PeterisP 3y ago

This is a bit of a chicken and egg problem, because the developers of these projects probably don't even have relevant AMD devices to test that, because why would they? The only reasonable way to break the cycle would be proactive intervention by AMD to contribute code and testing and easy "works out of the box" installation to all the major popular projects to add AMD support so they can sell more their hardware later, but I'm not seeing AMD doing that.

lainga 3y ago
Maybe the AMD device will just have to announce it's (KHTML, like Gecko).
mschuetz 3y ago
CUDA is just way to good in terms of UX with nothing platform-independent coming close to it. I wish there was a competitor, but there simply isn't. I wish AMD&Intel would extend compute shaders with pointers&pointer casting, arbitrary large buffers instead of just 4GB, device-wide sync, and function pointers. Those are kinda my must-have functionality. Even better, just use C++ for compute shaders.
- JonChesterfield 3y ago
  Freestanding C++ can be compiled to amdgpu or nvptx using clang. There are rough edges but it's usable.
JonChesterfield 3y ago
Like the pytorch one? Or the onnx? Or Triton? There's loads of these machine learning things, each of which built their thing assuming Nvidia, to which AMD is stoically working to implement support for their hardware. It's a task of unbounded scope with unclear return on investment and they're doing it anyway.

latchkey 3y ago

You didn't watch the announcement on the MI300x. You don't have to change that line to use AMD.

kaelinl 3y ago

I haven't used it personally, but my understanding is that AMD/ROCm-accelerated backends for PyTorch overload the "cuda" device and module so that ROCm shows up as CUDA for feature testing. They want to make a transition seamless, and lots of existing code checks for CUDA, so they do what's necessary to make that existing code run.

bragr 3y ago
Which tools actually pull down those packages though? If anything that's a counter point to the Intel CEO: industry is adopting CUDA as the defacto standard.
mappu 3y ago
That seems to be the case for me out-of-the-box. $ python3 -m venv . $ ./bin/pip3 install torch --index-url https://download.pytorch.org/whl/rocm5.6 $ ./bin/python3 -c 'import torch; print([torch.cuda.is_available(), torch.cuda.get_device_name(0)])' [True, 'AMD Radeon RX 6600 XT']

muxr 3y ago

zendnn is for CPUs. Pytorch ROCm version doesn't require you to change a single line of code. So all the same semantics like `device = "cuda"` work the same when using AMD GPUs.