4 ms·

I don't see it. Large customers are choosing AMD over nVidia. NVidia has a massive headstart, but there are diminishing returns to their software work. If AMD

I don't see it. Large customers are choosing AMD over nVidia. NVidia has a massive headstart, but there are diminishing returns to their software work. If AMD performance per dollar is 10% better at the hardware level, and their software performance is 90% of that of nvidia they are ahead.

WanderPanda 2y ago
Plus all the flexibility of CUDA is not needed for LLMs anyways
almostgotcaught 2y ago
Absolutely none of this comment is based on fact or reason. > I don't see it. Large customers are choosing AMD over nVidia. Please show me term sheets of contracts signed with "large customers". Please include net, gross, and recurring revenue. > diminishing returns to their software work Makes zero sense: software penetration follows power laws (network effects etc) not exponential decay laws. > If AMD performance per dollar is 10% better at the hardware level, and their software performance is 90% of that of nvidia they are ahead. Again: you cannot fathom how enormous that "if" there is so I highly recommend you actually go look at the impl to get a sense.
- roenxi 2y ago
  > Makes zero sense: software penetration follows power laws (network effects etc) not exponential decay laws. Software libraries aren't social networks, though. According to this argument we should all still be using C and not seeing new languages take hold. There is almost certainly a point where a library stops being able to add new value by being extended and that competitors can catch up to relatively easily no matter what the user demographics look like. There are definitely social aspects to software development so there might be some power laws to find; but building a competing ecosystem gets done regularly enough and works well (C# v. Java for example). Much easier to do than building a competing social network.
  - almostgotcaught 2y ago
    > Software libraries aren't social networks, though. They literally literally are - I follow people on GH in order to check out what they're doing and what repos they're starring/using. > Much easier to do than building a competing social network. We're talking about the ecosystem around the base stack remember? So no it's not easier when the bulk of the work needs to come from OSS contributors. People have the weirdest wishful thinking about this because they don't actually work in this area (and/or don't have stake in the game). If you're this confident please show me your portfolio allocation for NVDA and AMD. Otherwise hint: everyone working on this does not so casually dismiss NVIDIA's lead.
    - roenxi 2y ago
      GitHub is a social network. Libraries aren't. CUDA is not even on GitHub though as far as I recall. > We're talking about the ecosystem around the base stack remember? So no it's not easier when the bulk of the work needs to come from OSS contributors. The ecosystem exists as an abstraction around a base stack though. Much like saying the existence of Linux solidifies x86 dominance - that isn't what happens in practice, the Linux distros just pick up support for whatever hardware API are floating around. At some point the base can't add more value by adding more API features, and then the middleware starts supporting multiple bases depending on hardware economics. The OSS people have no incentive to lock themselves in to buying NVidia cards. If AMD cards tended to work for compute problems they'd probably much rather be using AMD. AMD has better support for anything OSS that isn't compute related.
      - almostgotcaught 2y ago
        I'll leave it at this: no one that works in this area agrees with you or the lowbrow/peanut gallery "NVIDIA has no moat" takes. If that doesn't give you pause then I guess you should just start a company and prove us all wrong.
        roenxi 2y ago
        > no one that works in this area agrees with you or the lowbrow/peanut gallery "NVIDIA has no moat" takes. Can you source that? That seems like a hard claim to source. Everyone in the area agrees that they'd rather be using Nvidia hardware. I'd rather be using Nvidia hardware, indeed I switched to a Nvidia card. But that is not because I see Nvidia as having a long term defensible position, but because AMD's compute drivers on consumer cards appear to suck. It was implementation details and AMD's weaknesses, not Nvidia's strengths. Important details, but nothing that can be realistically called a moat and certainly not anything to do with CUDA itself. The issues were far more foundational. Honestly it'd be interesting to have some real expert takes on what the problem with AMD is, because the major one I'm aware of was geohotz's and he didn't get much further than I did. Blockers came up much deeper in the stack than CUDA. ROCm would have been good enough except that it was running on AMD kernel drivers.
- Certhas 2y ago
  You have no facts and only cliche lines for reasons either. What does software penetrative or power laws have to do with anything? I don't know what relevance term sheets should have here, but AMD success at the HPC top end is pretty obvious if you look: https://top500.org/lists/top500/2024/06/ https://top500.org/lists/top500/2024/06/ AMD and nVidia are both powering about 1.5 Exaflop of compute of the top 10 supercomputer. Machines 1 and 5 are AMD, nVidia is number 3 and 6-10. They are still massively behind nVidia in the market overall: They are projecting a tenth of the datacenter revenues of nVidia. But that's still 4 billion in sales.
  - almostgotcaught 2y ago
    Hint: hpc (national labs) are not "large customers".
    - jfkfif 2y ago
      this good enough for you? https://www.amd.com/en/newsroom/press-releases/2024-5-21-amd-instinct-mi300x-accelerators-power-microsoft-a.html https://www.amd.com/en/newsroom/press-releases/2024-5-21-amd...
    - Certhas 2y ago
      Frontier has 38k MI250. Google and Amazon apparently bought 50k H100 in 2023. (Behind Meta and Microsoft at 150k) I believe 38k definitely qualifies as large. Maybe you misread me. I didn't mean to write that this was the default choice for large customers. Just that there are large customers for whom it makes sense to go AMD. The thing is, from where I am working and planning, nvidias cuda advantage is not a thriving community around it that would be hard to replicate. The community aspects are much more prominent higher up the stack, if you support pytorch and tensorflow you have a ton of community. Nvidia absolutely rocks at having high performance proprietary libraries for every niche use case. That's going to take time focus and investment, but the hand performance tuning of these libraries might only buy you so much over the pytorch version. Unless you are doing LLMs and pushing hard against the limits of what's possible, that last bit of tuning can wait. Nvidia had a call with us not long ago. I genuinely don'tsee the moat. If we manage to launch a product in our space it will run on anything pytorch runs. There is no advantage to marrying ourselves to cuda.

WanderPanda 2y ago

Plus all the flexibility of CUDA is not needed for LLMs anyways

almostgotcaught 2y ago

Absolutely none of this comment is based on fact or reason. > I don't see it. Large customers are choosing AMD over nVidia. Please show me term sheets of contracts signed with "large customers". Please include net, gross, and recurring revenue. > diminishing returns to their software work Makes zero sense: software penetration follows power laws (network effects etc) not exponential decay laws. > If AMD performance per dollar is 10% better at the hardware level, and their software performance is 90% of that of nvidia they are ahead. Again: you cannot fathom how enormous that "if" there is so I highly recommend you actually go look at the impl to get a sense.

roenxi 2y ago
> Makes zero sense: software penetration follows power laws (network effects etc) not exponential decay laws. Software libraries aren't social networks, though. According to this argument we should all still be using C and not seeing new languages take hold. There is almost certainly a point where a library stops being able to add new value by being extended and that competitors can catch up to relatively easily no matter what the user demographics look like. There are definitely social aspects to software development so there might be some power laws to find; but building a competing ecosystem gets done regularly enough and works well (C# v. Java for example). Much easier to do than building a competing social network.
- almostgotcaught 2y ago
  > Software libraries aren't social networks, though. They literally literally are - I follow people on GH in order to check out what they're doing and what repos they're starring/using. > Much easier to do than building a competing social network. We're talking about the ecosystem around the base stack remember? So no it's not easier when the bulk of the work needs to come from OSS contributors. People have the weirdest wishful thinking about this because they don't actually work in this area (and/or don't have stake in the game). If you're this confident please show me your portfolio allocation for NVDA and AMD. Otherwise hint: everyone working on this does not so casually dismiss NVIDIA's lead.
  - roenxi 2y ago
    GitHub is a social network. Libraries aren't. CUDA is not even on GitHub though as far as I recall. > We're talking about the ecosystem around the base stack remember? So no it's not easier when the bulk of the work needs to come from OSS contributors. The ecosystem exists as an abstraction around a base stack though. Much like saying the existence of Linux solidifies x86 dominance - that isn't what happens in practice, the Linux distros just pick up support for whatever hardware API are floating around. At some point the base can't add more value by adding more API features, and then the middleware starts supporting multiple bases depending on hardware economics. The OSS people have no incentive to lock themselves in to buying NVidia cards. If AMD cards tended to work for compute problems they'd probably much rather be using AMD. AMD has better support for anything OSS that isn't compute related.
    - almostgotcaught 2y ago
      I'll leave it at this: no one that works in this area agrees with you or the lowbrow/peanut gallery "NVIDIA has no moat" takes. If that doesn't give you pause then I guess you should just start a company and prove us all wrong.
      - roenxi 2y ago
        > no one that works in this area agrees with you or the lowbrow/peanut gallery "NVIDIA has no moat" takes. Can you source that? That seems like a hard claim to source. Everyone in the area agrees that they'd rather be using Nvidia hardware. I'd rather be using Nvidia hardware, indeed I switched to a Nvidia card. But that is not because I see Nvidia as having a long term defensible position, but because AMD's compute drivers on consumer cards appear to suck. It was implementation details and AMD's weaknesses, not Nvidia's strengths. Important details, but nothing that can be realistically called a moat and certainly not anything to do with CUDA itself. The issues were far more foundational. Honestly it'd be interesting to have some real expert takes on what the problem with AMD is, because the major one I'm aware of was geohotz's and he didn't get much further than I did. Blockers came up much deeper in the stack than CUDA. ROCm would have been good enough except that it was running on AMD kernel drivers.
Certhas 2y ago
You have no facts and only cliche lines for reasons either. What does software penetrative or power laws have to do with anything? I don't know what relevance term sheets should have here, but AMD success at the HPC top end is pretty obvious if you look: https://top500.org/lists/top500/2024/06/ https://top500.org/lists/top500/2024/06/ AMD and nVidia are both powering about 1.5 Exaflop of compute of the top 10 supercomputer. Machines 1 and 5 are AMD, nVidia is number 3 and 6-10. They are still massively behind nVidia in the market overall: They are projecting a tenth of the datacenter revenues of nVidia. But that's still 4 billion in sales.
- almostgotcaught 2y ago
  Hint: hpc (national labs) are not "large customers".
  - jfkfif 2y ago
    this good enough for you? https://www.amd.com/en/newsroom/press-releases/2024-5-21-amd-instinct-mi300x-accelerators-power-microsoft-a.html https://www.amd.com/en/newsroom/press-releases/2024-5-21-amd...
  - Certhas 2y ago
    Frontier has 38k MI250. Google and Amazon apparently bought 50k H100 in 2023. (Behind Meta and Microsoft at 150k) I believe 38k definitely qualifies as large. Maybe you misread me. I didn't mean to write that this was the default choice for large customers. Just that there are large customers for whom it makes sense to go AMD. The thing is, from where I am working and planning, nvidias cuda advantage is not a thriving community around it that would be hard to replicate. The community aspects are much more prominent higher up the stack, if you support pytorch and tensorflow you have a ton of community. Nvidia absolutely rocks at having high performance proprietary libraries for every niche use case. That's going to take time focus and investment, but the hand performance tuning of these libraries might only buy you so much over the pytorch version. Unless you are doing LLMs and pushing hard against the limits of what's possible, that last bit of tuning can wait. Nvidia had a call with us not long ago. I genuinely don'tsee the moat. If we manage to launch a product in our space it will run on anything pytorch runs. There is no advantage to marrying ourselves to cuda.