4 ms·

>their fab advantage might have let them really go after Nvidia. I think Cantrill spelled it out in the article, and I think people are overlooking (and for so

>their fab advantage might have let them really go after Nvidia. I think Cantrill spelled it out in the article, and I think people are overlooking (and for some reason really continue to overlook, or just have no idea) that nvidia's advantage is Software. I remember Larabee as a teenager - I just graduated high school, the forums were talking about it, and I had finally saved enough money to buy an 8800GTX. The 8800GTX was also coincidentally, the first consumer GPU to support CUDA. Almost 2 decades later and it feels like nvidia has an insurmountable moat in AI due to CUDA. This wasn't an accident, nvidia spent a ton of time, with developers, getting it right. FTA: >that NVIDIA had an indisputable lead, even in 2009; that for Intel to dominate NVIDIA it would have required conjuring software expertise and focus with which Intel has famously struggled Ignoring this piece is huge in claming that nvidia would be a quarter of the size. Can you even imagine Intel investing in a non-x86 developer platform for 15 years while they were in the lead?

MBCook 2y ago
Not the Intel we have. But if the graphics card stuff had really started to show promise, maybe they would have become an Intel that would. You’re dead right that Nvidia is where they are because of their software story and all the work they put into it. I don’t know if Intel could have learned that lesson.
- baq 2y ago
  > I don’t know if Intel could have learned that lesson. Intel invented this model with x86.
  - therealcamino 2y ago
    Really? It doesn't seem to me like much Intel-written software has been key to x86's success. The things I can remember from Intel are compilers, performance analysis tools, and threading and math libraries. But none of them were must-haves, or differentiated x86 chips from competing CPU architectures. Most of the software that made x86 successful came from third parties. And more recently, the reviews I read of the Archmage graphics drivers didn't make it sound like Intel had understood the importance of the software that was essential to a pretty big hardware release.
    - baq 2y ago
      Your examples are simply too recent. They've been doing it since the 70s: https://en.wikipedia.org/wiki/Intel_system_development_kit https://en.wikipedia.org/wiki/Intel_system_development_kit - that they forgot what made them the success they are is unfortunate, but also wholly self-inflicted.
      - therealcamino 2y ago
        Those are dev boards -- hardware, not software.
markhahn 2y ago
not sure why people give so much credit to Cuda, at least now. AI hasn't cared about Cuda for years - sure, Torch on NV will use it, but AI is all about Torch/TF, not details below that.
- rhaps0dy 2y ago
  What? This is really wrong. The name of the game these days is optimizing memory and FLOPs usage on these GPUs. For example, Hopper (H100) cards introduced the WGMMA (Warp-group Matrix-Multiply Accumulate) instruction, and anyone who does LLM training jumped to use it ASAP because without it you can't fully utilize the FLOPs. Anyone training AIs at the cutting age cares a lot about CUDA details. https://hazyresearch.stanford.edu/blog/2024-05-12-tk https://hazyresearch.stanford.edu/blog/2024-05-12-tk
  - hnaccount_rng 2y ago
    That’s not what OP meant though. That’s caring about _hardware_ capability. You could do that yourself for different hardware. And (OP’s point) any alternative hardware provider could do that for you. It just happens to be really, really hard
    - rhaps0dy 2y ago
      I think "AI is all about Torch/TF, not details below that" directly contradicts the fact that ML people very much care about the details, to squeeze performance out of the hardware.
      - nemothekid 2y ago
        >AI is all about Torch/TF, not details below that This could be 100% true, and the best ML engineers in the world could be completely oblivious to CUDA and it wouldn't change why nvidia is leading right now. Anyone who thinks it's "trivial" to use AMD cards should help geohot and his tinygrad boxes. Despite emailing the CEO several times, it was a clear uphill battle to get something working, and even now the sales page lists driver quality as "mediocre". If you open a GitHub issue or a forum post about your weird CUDA application that isn't working right, you will have an engineer help you. This is in nvidia's DNA and they did this before AI started printing money. On the flipside, you can email Lisa Su today and still not have problems fixed in AMDs drivers. I'm sure AMD is trying, this isn't to say they aren't, but having decent software culture at these large companies is hard. If AMD can't do it, I feel even less confident about Intel. If the CUDA moat truly didn't matter, we would see AMD cards in demand. However it doesn't and it isn't for lack of trying.

MBCook 2y ago

Not the Intel we have. But if the graphics card stuff had really started to show promise, maybe they would have become an Intel that would. You’re dead right that Nvidia is where they are because of their software story and all the work they put into it. I don’t know if Intel could have learned that lesson.

baq 2y ago
> I don’t know if Intel could have learned that lesson. Intel invented this model with x86.
- therealcamino 2y ago
  Really? It doesn't seem to me like much Intel-written software has been key to x86's success. The things I can remember from Intel are compilers, performance analysis tools, and threading and math libraries. But none of them were must-haves, or differentiated x86 chips from competing CPU architectures. Most of the software that made x86 successful came from third parties. And more recently, the reviews I read of the Archmage graphics drivers didn't make it sound like Intel had understood the importance of the software that was essential to a pretty big hardware release.
  - baq 2y ago
    Your examples are simply too recent. They've been doing it since the 70s: https://en.wikipedia.org/wiki/Intel_system_development_kit https://en.wikipedia.org/wiki/Intel_system_development_kit - that they forgot what made them the success they are is unfortunate, but also wholly self-inflicted.
    - therealcamino 2y ago
      Those are dev boards -- hardware, not software.

markhahn 2y ago

not sure why people give so much credit to Cuda, at least now. AI hasn't cared about Cuda for years - sure, Torch on NV will use it, but AI is all about Torch/TF, not details below that.

rhaps0dy 2y ago
What? This is really wrong. The name of the game these days is optimizing memory and FLOPs usage on these GPUs. For example, Hopper (H100) cards introduced the WGMMA (Warp-group Matrix-Multiply Accumulate) instruction, and anyone who does LLM training jumped to use it ASAP because without it you can't fully utilize the FLOPs. Anyone training AIs at the cutting age cares a lot about CUDA details. https://hazyresearch.stanford.edu/blog/2024-05-12-tk https://hazyresearch.stanford.edu/blog/2024-05-12-tk
- hnaccount_rng 2y ago
  That’s not what OP meant though. That’s caring about _hardware_ capability. You could do that yourself for different hardware. And (OP’s point) any alternative hardware provider could do that for you. It just happens to be really, really hard
  - rhaps0dy 2y ago
    I think "AI is all about Torch/TF, not details below that" directly contradicts the fact that ML people very much care about the details, to squeeze performance out of the hardware.
    - nemothekid 2y ago
      >AI is all about Torch/TF, not details below that This could be 100% true, and the best ML engineers in the world could be completely oblivious to CUDA and it wouldn't change why nvidia is leading right now. Anyone who thinks it's "trivial" to use AMD cards should help geohot and his tinygrad boxes. Despite emailing the CEO several times, it was a clear uphill battle to get something working, and even now the sales page lists driver quality as "mediocre". If you open a GitHub issue or a forum post about your weird CUDA application that isn't working right, you will have an engineer help you. This is in nvidia's DNA and they did this before AI started printing money. On the flipside, you can email Lisa Su today and still not have problems fixed in AMDs drivers. I'm sure AMD is trying, this isn't to say they aren't, but having decent software culture at these large companies is hard. If AMD can't do it, I feel even less confident about Intel. If the CUDA moat truly didn't matter, we would see AMD cards in demand. However it doesn't and it isn't for lack of trying.