4 ms·

> So there's no "PyTorch vs CUDA" because "PyTorch is CUDA". Not true. PyTorch has had an MPS backend[1] for Apple Silicon for quite some time now. And lat

> So there's no "PyTorch vs CUDA" because "PyTorch __is__ CUDA". Not true. PyTorch has had an MPS backend[1] for Apple Silicon for quite some time now. And lately, they've also added a HIP backend[2] for AMD hardware. I've written CUDA kernels for PyTorch in the past, it'd have been a lot easier if PyTorch were a thin wrapper over CUDA, but it isn't. [1]: https://pytorch.org/docs/stable/notes/mps.html https://pytorch.org/docs/stable/notes/mps.html [2]: https://pytorch.org/docs/stable/notes/hip.html https://pytorch.org/docs/stable/notes/hip.html

godelski 3y ago
Sure, I hope you can recognize that I was trying to convey that there were a lot of things. I mention ROCm because it is more than HIP and is what you'll see when you install pytorch. I hope you recognize that I was intending to convey that it was more than a wrapper. I'll admit that I could have phrased my intent much better, but I was trying to write to an audience of one with my best interpretation of their comment.
- weebull 3y ago
  You don't mention whether PyTorch runs with ROCm though, and it does. RX 6xxx and 7xxx series GPUs do pretty well with it.
  - godelski 3y ago
    > So there's no "PyTorch vs CUDA" because "PyTorch __is__ CUDA". ==> You can also say it's a lot of other things like ROCm. <== I mean my intent was just to communicate that PyTorch runs CUDA in the backend, which was what I meant by API, because it seemed the person I was responding to wasn't aware of this backend code you're interfacing with. I thought adding the same phrasing about it also "being" ROCm and using "such as" and "other things," that others would have allowed anyone with more knowledge to infer that I'm aware of the generalization but I guess I was deeply mistaken given the replies that are certain that I can't read the pytorch homepage. Though I'm not quite sure how you came to the conclusion that I wasn't aware of ROCm support since I did explicitly mention it. Do you have feedback for how I can better communicate? I seem to be running into failures like this a lot lately where I feel like I can point to where I clearly stated something but that doesn't matter because communication is about getting the person on the other side to receive the message. I've been scratching my head about this and could definitely use the outside perspective.
    - weebull 3y ago
      "PyTorch uses the best backend it has available. If that's CUDA, it will use CUDA. It will also use ROCm on AMD GPUs and optimised BLAS libraries on CPUs. These backeds are selected at compile time though, so you need to install a version of PyTorch with optimised support for your hardware. "

> So there's no "PyTorch vs CUDA" because "PyTorch __is__ CUDA". Not true. PyTorch has had an MPS backend[1] for Apple Silicon for quite some time now. And lat

godelski 3y ago

Sure, I hope you can recognize that I was trying to convey that there were a lot of things. I mention ROCm because it is more than HIP and is what you'll see when you install pytorch. I hope you recognize that I was intending to convey that it was more than a wrapper. I'll admit that I could have phrased my intent much better, but I was trying to write to an audience of one with my best interpretation of their comment.

weebull 3y ago
You don't mention whether PyTorch runs with ROCm though, and it does. RX 6xxx and 7xxx series GPUs do pretty well with it.
- godelski 3y ago
  > So there's no "PyTorch vs CUDA" because "PyTorch __is__ CUDA". ==> You can also say it's a lot of other things like ROCm. <== I mean my intent was just to communicate that PyTorch runs CUDA in the backend, which was what I meant by API, because it seemed the person I was responding to wasn't aware of this backend code you're interfacing with. I thought adding the same phrasing about it also "being" ROCm and using "such as" and "other things," that others would have allowed anyone with more knowledge to infer that I'm aware of the generalization but I guess I was deeply mistaken given the replies that are certain that I can't read the pytorch homepage. Though I'm not quite sure how you came to the conclusion that I wasn't aware of ROCm support since I did explicitly mention it. Do you have feedback for how I can better communicate? I seem to be running into failures like this a lot lately where I feel like I can point to where I clearly stated something but that doesn't matter because communication is about getting the person on the other side to receive the message. I've been scratching my head about this and could definitely use the outside perspective.
  - weebull 3y ago
    "PyTorch uses the best backend it has available. If that's CUDA, it will use CUDA. It will also use ROCm on AMD GPUs and optimised BLAS libraries on CPUs. These backeds are selected at compile time though, so you need to install a version of PyTorch with optimised support for your hardware. "