6 ms·

One word. CUDA

justinjlynn 3y ago
Valve's Proton/wine... Apple's M5 cpu. Software/hardware layers, especially system software and hardware interfaces that must remain stable and compatible essentially forever, aren't the strong lock in mechanisms everyone thinks they are.
- m00x 3y ago
  That absolutely doesn't work in ML training.
  - imtringued 3y ago
    It would work if AMD focused on ROCm support for their consumer GPUs. The reason for that is quite simple. Angry consumers using your GPUs produces a lot of data that helps you design future GPUs and improve the drivers.
  - jjoonathan 3y ago
    It absolutely does, but AMD's execution on this front has been unmitigated dogshit for the last decade and now every engineer in this niche has a scar or five from giving AMD chance after chance after chance and then limping back to NVDA when the slog is just too much. Now that they have money, I'm hoping they can turn this around. I hear that they have... but I've always heard that and it has never been true. I need to see to believe, now.
    - Pet_Ant 3y ago
      Are you referring to OpenCL?
      - imtringued 3y ago
        https://www.phoronix.com/news/OpenCL-On-Mesa-Matrix https://www.phoronix.com/news/OpenCL-On-Mesa-Matrix
    - aunty_helen 3y ago
      AMD's valuation is 1/5th of Nvidia now. They've done amazing in the last 5 years in the CPU market but have absolutely missed the boat on a _trillion_ dollar valuation because they didn't put enough effort into the GPGPU market. This is all the while being 1 of 2 companies in the world that was best positioned to capitalise. I like Dr. Su, she's an amazing CEO and has stuck it to Intel. But Intel wasn't the one they should've been spending all their effort on. Hopefully they realise this because Nvidia selling GPUs for 40k can only go on for so long.
      - Drybones 3y ago
        I disagree. The market ready for disruption was the CPU market stagnated by Intel. It was also the perfect opportunity to put the limited financial resources behind to come out with a viable product to compete that didn't need a ton of software ecosystem to work. Datacenters, gamers, Cloud hyperscalers, Supercomputers, they all needed something to replace their aging and inefficient Intel Skylake Xeons and Core CPUs. And their chiplet tech made it possible to make this CPU product span the entire portfolio for cheap. If they decided to compete with GPU, they would have just lost in gaming GPU sales to NVIDIA due to mindshare and they wouldn't have had the resources to develop the software to compete. And datacenter GPUs were still a niche and rare product. AI/ML was not realistically something they could predict or bet the house on 7 years ago. And just like Lisa said in the recent presentation, AI is in its early stages and will be a growing and long term business venture. Additionally, their CPU chiplet developments were critical in producing the talent and experience that would translate to GPU chiplets that AMD is not utilizing on RDNA 3 and CDNA 3, providing a strategic advantage over NVIDIA. They still have the time to enter it with their MI300s AND now they have the money and resources to develop their software ecosystems more. AMD absolutely made the right move to focus on Zen and HPC. It's not their fault that investors are blindly overhyped about AI. AMD's greatest threat in datacenter AI hardware isn't even NVIDIA. It's the biggest tech companies producing their own AI hardware (Google, Meta, Tesla, Amazon) effectively and eventually eliminating the need for AMD or NVIDIA GPU/AI hardware.
        aunty_helen 3y ago
        I appreciate your points but you're being generous to AMD. They bought ATI in 2006. I remember in 2007 seeing CUDA for the first time in the password cracking scene and thinking wow they've done something amazing. OpenCL was there too, the "one framework heterogenious processing" sounded amazing but quickly became the ugly cousin. Then in 2011 with crypto, once again Nvidia was always supported but ATI was that other case that required the different install with only some support. Then 6 years ago when I started working professionally in ai, it was CUDA only for most of the applications. AMD had some stuff but had pretty much given up on OpenCL and at this point was a distant second. If you chose AMD you were quickly going to be locked out while the cool kids played with CUDA and TF. This was in a time when there may have only been one framework or library to do a particular algo. So it really was a lockout. So to your point, 16 years ago when I first saw GPGPU, you could bet your house on it becoming something massive. The scientific applications alone were obvious to anyone with a copy of BOINC. Nvidia have shown a masterclass of building something as a corporate over many years and really dominating all competition. AMD should have jumped onboard with TF and made sure any CUDA enabled algo had a _insert whatever AMD would have used_ equivalent. But they didn't, they couldn't even get linux drivers to work.
  - wongarsu 3y ago
    ML training sounds like the easiest part. All that would be needed on that front would be AMD engineers writing good AMD backends for pytorch and tensorflow. A much simpler task than offering an optimized general-purpose interface and getting people to use it (whether that's optimized OpenCL, a CUDA-compatible API or something else). Of course so far AMD has done a laughably bad job at these kinds of things. I guess there's hope
Fnoord 3y ago
CUDA is proprietary, you're forced to use Nvidia GPUs/TPUs on it (would Coral work?). I bought a couple of Jetson for this purpose. I'm forced to run Ubuntu 20.04 LTS on them. But it works. My workstation however can run on FOSS AMD64 and AMD/ATI GPU. Of course, the CPU isn't FOSS (RISC-V could be, but GPU not). But at least the drivers are, and they're a better netizen than Intel, and Raptor Engineering is too expensive for me.

One word. CUDA

Valve's Proton/wine... Apple's M5 cpu. Software/hardware layers, especially system software and hardware interfaces that must remain stable and compatible essentially forever, aren't the strong lock in mechanisms everyone thinks they are.

m00x 3y ago
That absolutely doesn't work in ML training.
- imtringued 3y ago
  It would work if AMD focused on ROCm support for their consumer GPUs. The reason for that is quite simple. Angry consumers using your GPUs produces a lot of data that helps you design future GPUs and improve the drivers.
- jjoonathan 3y ago
  It absolutely does, but AMD's execution on this front has been unmitigated dogshit for the last decade and now every engineer in this niche has a scar or five from giving AMD chance after chance after chance and then limping back to NVDA when the slog is just too much. Now that they have money, I'm hoping they can turn this around. I hear that they have... but I've always heard that and it has never been true. I need to see to believe, now.
  - Pet_Ant 3y ago
    Are you referring to OpenCL?
    - imtringued 3y ago
      https://www.phoronix.com/news/OpenCL-On-Mesa-Matrix https://www.phoronix.com/news/OpenCL-On-Mesa-Matrix
  - aunty_helen 3y ago
    AMD's valuation is 1/5th of Nvidia now. They've done amazing in the last 5 years in the CPU market but have absolutely missed the boat on a _trillion_ dollar valuation because they didn't put enough effort into the GPGPU market. This is all the while being 1 of 2 companies in the world that was best positioned to capitalise. I like Dr. Su, she's an amazing CEO and has stuck it to Intel. But Intel wasn't the one they should've been spending all their effort on. Hopefully they realise this because Nvidia selling GPUs for 40k can only go on for so long.
    - Drybones 3y ago
      I disagree. The market ready for disruption was the CPU market stagnated by Intel. It was also the perfect opportunity to put the limited financial resources behind to come out with a viable product to compete that didn't need a ton of software ecosystem to work. Datacenters, gamers, Cloud hyperscalers, Supercomputers, they all needed something to replace their aging and inefficient Intel Skylake Xeons and Core CPUs. And their chiplet tech made it possible to make this CPU product span the entire portfolio for cheap. If they decided to compete with GPU, they would have just lost in gaming GPU sales to NVIDIA due to mindshare and they wouldn't have had the resources to develop the software to compete. And datacenter GPUs were still a niche and rare product. AI/ML was not realistically something they could predict or bet the house on 7 years ago. And just like Lisa said in the recent presentation, AI is in its early stages and will be a growing and long term business venture. Additionally, their CPU chiplet developments were critical in producing the talent and experience that would translate to GPU chiplets that AMD is not utilizing on RDNA 3 and CDNA 3, providing a strategic advantage over NVIDIA. They still have the time to enter it with their MI300s AND now they have the money and resources to develop their software ecosystems more. AMD absolutely made the right move to focus on Zen and HPC. It's not their fault that investors are blindly overhyped about AI. AMD's greatest threat in datacenter AI hardware isn't even NVIDIA. It's the biggest tech companies producing their own AI hardware (Google, Meta, Tesla, Amazon) effectively and eventually eliminating the need for AMD or NVIDIA GPU/AI hardware.
      - aunty_helen 3y ago
        I appreciate your points but you're being generous to AMD. They bought ATI in 2006. I remember in 2007 seeing CUDA for the first time in the password cracking scene and thinking wow they've done something amazing. OpenCL was there too, the "one framework heterogenious processing" sounded amazing but quickly became the ugly cousin. Then in 2011 with crypto, once again Nvidia was always supported but ATI was that other case that required the different install with only some support. Then 6 years ago when I started working professionally in ai, it was CUDA only for most of the applications. AMD had some stuff but had pretty much given up on OpenCL and at this point was a distant second. If you chose AMD you were quickly going to be locked out while the cool kids played with CUDA and TF. This was in a time when there may have only been one framework or library to do a particular algo. So it really was a lockout. So to your point, 16 years ago when I first saw GPGPU, you could bet your house on it becoming something massive. The scientific applications alone were obvious to anyone with a copy of BOINC. Nvidia have shown a masterclass of building something as a corporate over many years and really dominating all competition. AMD should have jumped onboard with TF and made sure any CUDA enabled algo had a _insert whatever AMD would have used_ equivalent. But they didn't, they couldn't even get linux drivers to work.
- wongarsu 3y ago
  ML training sounds like the easiest part. All that would be needed on that front would be AMD engineers writing good AMD backends for pytorch and tensorflow. A much simpler task than offering an optimized general-purpose interface and getting people to use it (whether that's optimized OpenCL, a CUDA-compatible API or something else). Of course so far AMD has done a laughably bad job at these kinds of things. I guess there's hope

Fnoord 3y ago

CUDA is proprietary, you're forced to use Nvidia GPUs/TPUs on it (would Coral work?). I bought a couple of Jetson for this purpose. I'm forced to run Ubuntu 20.04 LTS on them. But it works. My workstation however can run on FOSS AMD64 and AMD/ATI GPU. Of course, the CPU isn't FOSS (RISC-V could be, but GPU not). But at least the drivers are, and they're a better netizen than Intel, and Raptor Engineering is too expensive for me.