4 ms·

For many years, AMD GPUs were cheaper and equally or more performant than NVIDIA GPUs. Did that stop NVIDIAs lead in AI training? No, software support is the ke

For many years, AMD GPUs were cheaper and equally or more performant than NVIDIA GPUs. Did that stop NVIDIAs lead in AI training? No, software support is the key component. It's the same with Apple's M1. I don't know anyone who is using it. People need obscure x64 instructions and CUDA. And that means you buy AMD CPU + NVIDIA GPU as the cheapest high-performance combination. Also, I expect that the 5nm process is responsible for a large part of the power savings in the M1. So I would expect an AMD CPU on TSMCs 5nm to be pretty close in terms of performance per watt. It's just that the mainstream market isn't willing to pay the premium price for 5nm yet. So AMD produces "good enough" at a price that people are happy to pay, which means they need to hold off on going 5nm for now.

SaulJLH 4y ago
"For many years, AMD GPUs were cheaper and equally or more performant than NVIDIA GPUs. Did that stop NVIDIAs lead in AI training? No, software support is the key component. It's the same with Apple's M1. I don't know anyone who is using it. People need obscure x64 instructions and CUDA. And that means you buy AMD CPU + NVIDIA GPU as the cheapest high-performance combination." Riiight, but, OP isn't necessarily talking purely from an AI/ML use case.
- fxtentacle 4y ago
  Well if you don't need the high performance, then I'd expect people will go for a cheap CPU, meaning no M1 Ultra.
  - pclmulqdq 4y ago
    ML training isn't the only application of high performance computing. Apple is making video editing machines - which is why they have the mix of cores that they do. Simulations are also high-performance computing applications.
    - PaulDavisThe1st 4y ago
      > Apple is making video editing machines - which is why they have the mix of cores that they do. Nope. They have that mix because they've decided to prioritize power consumption reduction (I suspect because they sell way, way more laptops that desktops/servers). It's actually a PITA for developers (you have to ensure that your compute-heavy threads are always on the performance cores), and conceptually confusing for users if they ever have to confront it. For pure performance, with no concern for power consumption, there are faster solutions.
tambourine_man 4y ago
Software changes much faster these days. We are all dealing with high abstraction layers and legacy x86/64 and even CUDA can be replaced without much disruption in most cases. If it’s proven that Apple can consistently provide better performance per watt per dollar and have enough chips available that people can actually buy, I think they can stand a chance to gain the AI market.
- fxtentacle 4y ago
  "even CUDA can be replaced without much disruption in most cases" Tell that to AMD who's been missing out on the very lucrative government data-center market because they can't replace CUDA well enough. BTW, PyTorch and TensorFlow still ship without AMD support despite AMD's ROCm being around for 6 years now.
  - pca006132 4y ago
    iirc there are comments saying that ROCm is not very mature yet. I wish OpenCL can evolve to compete with CUDA
  - tambourine_man 4y ago
    Is that more an indictment of ROCm immaturity or of PyTorch/TensorFlow unwillingness to support it?
    - fxtentacle 4y ago
      I believe ROCm isn't usable yet. I don't know anyone who successfully trained a state of the art neural network on anything that was not NVIDIA. AMD keeps offering their own forks of PyTorch and TensorFlow, but last time I checked they were years outdated, and nobody uses them. So either PyTorch/TensorFlow are secretly sponsored by NVIDIA, or AMD's and Apple's CUDA replacements aren't mature enough yet. Given how many bugs there still are in CUDA despite them being around for a long time, I'd guess that building a GPU-capable BLAS library is just a lot more work than one would think. I mean I also always wonder why CUDA comes with 400 MB of GPU shaders ... but apparently they need that to cover every edge case in a fast way. FYI, this is the ROCm "getting started" guide: https://github.com/RadeonOpenCompute/ROCm-docker/blob/master/quick-start.md https://github.com/RadeonOpenCompute/ROCm-docker/blob/master... NVIDIA has a GUI installer for CUDA.
paulmd 4y ago
> It's the same with Apple's M1. I don't know anyone who is using it. "I can't believe Nixon won. I don't know anyone who voted for him." Yes, a misquotation, but still. The Java/JVM world has dived into it. Java is all JVM code and it's platform-independent, and unlike, say, Python, the Java world never dived that heavily into native-interface code that tied them to a specific instruction set, they built a faster JVM instead of going out to native code. M1 is so good you can get better performance on a M1 MBA with significantly longer battery life compared to an i9 MBP of the previous generation. Same for average users who don't do any of the developer things - HN itself is a bubble and most users don't know or care the uarch is different at all, they just know a base-tier M1 feels better than an i9, and they're right. > So I would expect an AMD CPU on TSMCs 5nm to be pretty close in terms of performance per watt. It's just that the mainstream market isn't willing to pay the premium price for 5nm yet Apple and AMD are going to be on the same node this year.

SaulJLH 4y ago

"For many years, AMD GPUs were cheaper and equally or more performant than NVIDIA GPUs. Did that stop NVIDIAs lead in AI training? No, software support is the key component. It's the same with Apple's M1. I don't know anyone who is using it. People need obscure x64 instructions and CUDA. And that means you buy AMD CPU + NVIDIA GPU as the cheapest high-performance combination." Riiight, but, OP isn't necessarily talking purely from an AI/ML use case.

fxtentacle 4y ago
Well if you don't need the high performance, then I'd expect people will go for a cheap CPU, meaning no M1 Ultra.
- pclmulqdq 4y ago
  ML training isn't the only application of high performance computing. Apple is making video editing machines - which is why they have the mix of cores that they do. Simulations are also high-performance computing applications.
  - PaulDavisThe1st 4y ago
    > Apple is making video editing machines - which is why they have the mix of cores that they do. Nope. They have that mix because they've decided to prioritize power consumption reduction (I suspect because they sell way, way more laptops that desktops/servers). It's actually a PITA for developers (you have to ensure that your compute-heavy threads are always on the performance cores), and conceptually confusing for users if they ever have to confront it. For pure performance, with no concern for power consumption, there are faster solutions.

tambourine_man 4y ago

Software changes much faster these days. We are all dealing with high abstraction layers and legacy x86/64 and even CUDA can be replaced without much disruption in most cases. If it’s proven that Apple can consistently provide better performance per watt per dollar and have enough chips available that people can actually buy, I think they can stand a chance to gain the AI market.

fxtentacle 4y ago
"even CUDA can be replaced without much disruption in most cases" Tell that to AMD who's been missing out on the very lucrative government data-center market because they can't replace CUDA well enough. BTW, PyTorch and TensorFlow still ship without AMD support despite AMD's ROCm being around for 6 years now.
- pca006132 4y ago
  iirc there are comments saying that ROCm is not very mature yet. I wish OpenCL can evolve to compete with CUDA
- tambourine_man 4y ago
  Is that more an indictment of ROCm immaturity or of PyTorch/TensorFlow unwillingness to support it?
  - fxtentacle 4y ago
    I believe ROCm isn't usable yet. I don't know anyone who successfully trained a state of the art neural network on anything that was not NVIDIA. AMD keeps offering their own forks of PyTorch and TensorFlow, but last time I checked they were years outdated, and nobody uses them. So either PyTorch/TensorFlow are secretly sponsored by NVIDIA, or AMD's and Apple's CUDA replacements aren't mature enough yet. Given how many bugs there still are in CUDA despite them being around for a long time, I'd guess that building a GPU-capable BLAS library is just a lot more work than one would think. I mean I also always wonder why CUDA comes with 400 MB of GPU shaders ... but apparently they need that to cover every edge case in a fast way. FYI, this is the ROCm "getting started" guide: https://github.com/RadeonOpenCompute/ROCm-docker/blob/master/quick-start.md https://github.com/RadeonOpenCompute/ROCm-docker/blob/master... NVIDIA has a GUI installer for CUDA.

paulmd 4y ago

> It's the same with Apple's M1. I don't know anyone who is using it. "I can't believe Nixon won. I don't know anyone who voted for him." Yes, a misquotation, but still. The Java/JVM world has dived into it. Java is all JVM code and it's platform-independent, and unlike, say, Python, the Java world never dived that heavily into native-interface code that tied them to a specific instruction set, they built a faster JVM instead of going out to native code. M1 is so good you can get better performance on a M1 MBA with significantly longer battery life compared to an i9 MBP of the previous generation. Same for average users who don't do any of the developer things - HN itself is a bubble and most users don't know or care the uarch is different at all, they just know a base-tier M1 feels better than an i9, and they're right. > So I would expect an AMD CPU on TSMCs 5nm to be pretty close in terms of performance per watt. It's just that the mainstream market isn't willing to pay the premium price for 5nm yet Apple and AMD are going to be on the same node this year.