3 ms·

They’re going to sell a billion dollars of GPUs to a handful of customers while NVIDIA sells a trillion dollars of their products to everyone. Every framework,

They’re going to sell a billion dollars of GPUs to a handful of customers while NVIDIA sells a trillion dollars of their products to everyone. Every framework, library, demo, tool, and app is going to use CUDA forever and ever while some “account manager” at AMD takes a government procurement officer to lunch to sell one more supercomputer that year.

kllrnohj 2y ago
Nvidia's 2024 data center revenue was $46B. They got a long fucking way to go to get to trillion dollars of product.
- jiggawatts 2y ago
  Take a look at this chart going back ~3Y: https://ycharts.com/indicators/nvidia_corp_nvda_data_center_revenue_quarterly https://ycharts.com/indicators/nvidia_corp_nvda_data_center_... Their quarterly data centre revenue is now $22.6B! Even assuming that it immediately levels off, that's $90B over the next 12 months. If it merely doubles, then they'll hit a total of $1T in revenue in about 6 years. I'm an AI pessimist. The current crop of generative LLMs are cute, but not a direct replacement for humans in all but a few menial tasks. However, there's a very wide range of algorithmic improvements available, which wouldn't have been explored three years ago. Nobody had the funding, motivation, or hardware. Suddenly, everyone believes that it is possible, and everyone is throwing money at the problem. Even if the fruits of all of this investment is just a ~10% improvement in business productivity, that's easily worth $1T to the world economy over the next decade or so. AMD is absolutely leaving trillions of dollars on the table because they're too comfortable selling one supercomputer at a time to government customers. Those customers will stop buying their kit very soon, because all of the useful software is being written for CUDA only.
  - kllrnohj 2y ago
    Did you look at your own chart? There's no trend of 200% growth. Rather this last few quarters were a huge jump from relatively modest gains the years prior. Expecting 6 years of "merely doubling" is absolutely bonkers lol Who can even afford to buy that much product? Are you expecting Apple, Microsoft, Alphabet, Amazon, etc to all dump 100% of their cash on Nvidia GPUs? Even then that doesn't get you to a trillion dollars
    - jiggawatts 2y ago
      Meta alone bought 350,000 H100 GPUs, which cost them $10.5 billion: https://www.pcmag.com/news/zuckerbergs-meta-is-spending-billions-to-buy-350000-nvidia-h100-gpus https://www.pcmag.com/news/zuckerbergs-meta-is-spending-bill... This kind of AI capital investment seems to have helped them improve the feed recommendations, doubling their market cap over the last few years. In other words, they got their money back many times over! Chances are that they're going to invest this capital into B100 GPUs next year. Apple is about to revamp Siri with generative AI for hundreds of millions of their customers. I don't know how many GPUs that'll require, but I assume... many. There's a gold rush, and NVIDIA is the only shovel manufacturer in the world right now.
      - kllrnohj 2y ago
        > Meta alone bought 350,000 H100 GPUs, which cost them $10.5 billion Right, which means you need about a trillion dollars more to get to a trillion dollars. There's not another 100 Metas floating around. > Apple is about to revamp Siri with generative AI for hundreds of millions of their customers. I don't know how many GPUs that'll require, but I assume... many. Apple also said they were doing it with their silicon. Apple in particular is all but guaranteed to refuse to buy from Nvidia even. > There's a gold rush, and NVIDIA is the only shovel manufacturer in the world right now. lol no they aren't. This is literally a post about AMD's AI product even. But Apple and Google both have in-house chips as well. Nvidia is the big general party player, for sure, but they aren't the only. And more to the point, exponential growth of the already largest player for 6 years is still fucking absurd.
        jiggawatts 2y ago
        The GDP of the US alone over the next five years is $135T. Throw in other modern economies that use cloud services like Office 365 and you’re over $200T. If AI can improve productivity by just 1% then that is $2T more. If it costs $1T in NVIDIA hardware then this is well worth it.
        roenxi 2y ago
HarHarVeryFunny 2y ago
I'd guess that the majority of ML software is written in PyTorch, not in CUDA, and PyTorch has support for multiple backends including AMD. torch.compile also supports AMD (generating Triton kernels, same as it does for NVIDIA), so for most people there's no need to go lower level.
- pjmlp 2y ago
  GPUs are used for more than only ML workloads. CUDA relevance in the industry is so big now, that NVidia has several WG21 seats, and helps driving heterogenous programming roadmap for C++.
  - HarHarVeryFunny 2y ago
    You can use PyTorch for more than ML. No need to use backprop. Thinks of it as GPU accelerated NumPy.
    - pjmlp 2y ago
      I would like to see OctaneRender done in Pytorch. /s
      - HarHarVeryFunny 2y ago
        Sure, but if the OctaneRender folk wanted to support AMD, then I highly doubt they'd be interested in a CUDA compatability layer either - they'd want to be using the lowest level API possible (Vulkan?) to get close to the metal and optimize performance.
        pjmlp 2y ago
        See, that is where you got all wrong, they dropped Vulkan for CUDA, and even made a talk about it at GTC. https://www.cgchannel.com/2023/11/otoy-releases-first-public-preview-of-octanerender-2024-1/ https://www.cgchannel.com/2023/11/otoy-releases-first-public... https://www.cgchannel.com/2023/11/otoy-unveils-the-octane-2024-product-roadmap/ https://www.cgchannel.com/2023/11/otoy-unveils-the-octane-20... And then again, there are plenty of other cases where Pytorch makes absolute no sense in GPU, which was the whole starting point.

kllrnohj 2y ago

Nvidia's 2024 data center revenue was $46B. They got a long fucking way to go to get to trillion dollars of product.

jiggawatts 2y ago
Take a look at this chart going back ~3Y: https://ycharts.com/indicators/nvidia_corp_nvda_data_center_revenue_quarterly https://ycharts.com/indicators/nvidia_corp_nvda_data_center_... Their quarterly data centre revenue is now $22.6B! Even assuming that it immediately levels off, that's $90B over the next 12 months. If it merely doubles, then they'll hit a total of $1T in revenue in about 6 years. I'm an AI pessimist. The current crop of generative LLMs are cute, but not a direct replacement for humans in all but a few menial tasks. However, there's a very wide range of algorithmic improvements available, which wouldn't have been explored three years ago. Nobody had the funding, motivation, or hardware. Suddenly, everyone believes that it is possible, and everyone is throwing money at the problem. Even if the fruits of all of this investment is just a ~10% improvement in business productivity, that's easily worth $1T to the world economy over the next decade or so. AMD is absolutely leaving trillions of dollars on the table because they're too comfortable selling one supercomputer at a time to government customers. Those customers will stop buying their kit very soon, because all of the useful software is being written for CUDA only.
- kllrnohj 2y ago
  Did you look at your own chart? There's no trend of 200% growth. Rather this last few quarters were a huge jump from relatively modest gains the years prior. Expecting 6 years of "merely doubling" is absolutely bonkers lol Who can even afford to buy that much product? Are you expecting Apple, Microsoft, Alphabet, Amazon, etc to all dump 100% of their cash on Nvidia GPUs? Even then that doesn't get you to a trillion dollars
  - jiggawatts 2y ago
    Meta alone bought 350,000 H100 GPUs, which cost them $10.5 billion: https://www.pcmag.com/news/zuckerbergs-meta-is-spending-billions-to-buy-350000-nvidia-h100-gpus https://www.pcmag.com/news/zuckerbergs-meta-is-spending-bill... This kind of AI capital investment seems to have helped them improve the feed recommendations, doubling their market cap over the last few years. In other words, they got their money back many times over! Chances are that they're going to invest this capital into B100 GPUs next year. Apple is about to revamp Siri with generative AI for hundreds of millions of their customers. I don't know how many GPUs that'll require, but I assume... many. There's a gold rush, and NVIDIA is the only shovel manufacturer in the world right now.
    - kllrnohj 2y ago
      > Meta alone bought 350,000 H100 GPUs, which cost them $10.5 billion Right, which means you need about a trillion dollars more to get to a trillion dollars. There's not another 100 Metas floating around. > Apple is about to revamp Siri with generative AI for hundreds of millions of their customers. I don't know how many GPUs that'll require, but I assume... many. Apple also said they were doing it with their silicon. Apple in particular is all but guaranteed to refuse to buy from Nvidia even. > There's a gold rush, and NVIDIA is the only shovel manufacturer in the world right now. lol no they aren't. This is literally a post about AMD's AI product even. But Apple and Google both have in-house chips as well. Nvidia is the big general party player, for sure, but they aren't the only. And more to the point, exponential growth of the already largest player for 6 years is still fucking absurd.
      - jiggawatts 2y ago
        The GDP of the US alone over the next five years is $135T. Throw in other modern economies that use cloud services like Office 365 and you’re over $200T. If AI can improve productivity by just 1% then that is $2T more. If it costs $1T in NVIDIA hardware then this is well worth it.
        roenxi 2y ago

HarHarVeryFunny 2y ago

I'd guess that the majority of ML software is written in PyTorch, not in CUDA, and PyTorch has support for multiple backends including AMD. torch.compile also supports AMD (generating Triton kernels, same as it does for NVIDIA), so for most people there's no need to go lower level.

pjmlp 2y ago
GPUs are used for more than only ML workloads. CUDA relevance in the industry is so big now, that NVidia has several WG21 seats, and helps driving heterogenous programming roadmap for C++.
- HarHarVeryFunny 2y ago
  You can use PyTorch for more than ML. No need to use backprop. Thinks of it as GPU accelerated NumPy.
  - pjmlp 2y ago
    I would like to see OctaneRender done in Pytorch. /s
    - HarHarVeryFunny 2y ago
      Sure, but if the OctaneRender folk wanted to support AMD, then I highly doubt they'd be interested in a CUDA compatability layer either - they'd want to be using the lowest level API possible (Vulkan?) to get close to the metal and optimize performance.
      - pjmlp 2y ago
        See, that is where you got all wrong, they dropped Vulkan for CUDA, and even made a talk about it at GTC. https://www.cgchannel.com/2023/11/otoy-releases-first-public-preview-of-octanerender-2024-1/ https://www.cgchannel.com/2023/11/otoy-releases-first-public... https://www.cgchannel.com/2023/11/otoy-unveils-the-octane-2024-product-roadmap/ https://www.cgchannel.com/2023/11/otoy-unveils-the-octane-20... And then again, there are plenty of other cases where Pytorch makes absolute no sense in GPU, which was the whole starting point.