4 ms·

This is a really great breakdown. With TPUs seemingly more efficient and costing less overall, how does this play for Nvidia? What's to stop them from entering

by binkHN 10mo ago

This is a really great breakdown. With TPUs seemingly more efficient and costing less overall, how does this play for Nvidia? What's to stop them from entering the TPU race with their $5 trillion valuation?

sysguest 10mo ago
$5 trillion valuation doesn't mean it has $5 trillion cash in pocket -- so "it depends"
dragonwriter 10mo ago
> What's to stop them from entering the TPU race with their $5 trillion valuation? Valuation isn’t available money; they'd have to raise more money in the current, probably tighter for them, investment environment to enter the TPU race, since the money they have already raised that that valuation is based on is already needed to provide runway for what they are already doing without putting money into the TPU race
matwood 10mo ago
As others mentioned, 5T isn't money available to NVDA. It could leverage that to buy a TPU company in an all stock deal though. The bigger issue is that entering a 'race' implies a race to the bottom. I've noted this before, but one of NVDA's biggest risks is that its primary customers are also technical, also make hardware, also have money, and clearly see NVDA's margin (70% gross!!, 50%+ profit) as something they want to eliminate. Google was first to get there (not a surprise), but Meta is also working on its own hardware along with Amazon. This isn't a doom post for NVDA the company, but its stock price is riding a knifes edge. Any margin or growth contraction will not be a good day for their stock or the S&P.
- sigmoid10 10mo ago
  Making the hardware is actually the easy part. Everyone and their uncle who had some cash have tried by now: Microsoft, Meta, Tesla, Huawei, Amazon, Intel - the list goes on and on. But Nvidia is not a chip company. Huang himself said they are mostly a software company. And that is how they were able to build a gigantic moat. Because noone else has even come close on the software side. Google is the only one who has had some success on this side, because they also spent tons of money and time on software refinement by now, while all the other chips vanished into obscurity.
  - sanjayjc 10mo ago
    Genuine question: given LLMs' inexorable commoditization of software, how soon before NVDA's CUDA moat is breached too? Is CUDA somehow fundamentally different from other kinds of software or firmware?
    - tomrod 10mo ago
      Current Gen LLMs are not breaching the moat yet.
      - fzzzy 10mo ago
        Yeah they are. llama.cpp has had good performance on cpu, amd, and apple metal for at least a year now.
        tomrod 10mo ago
        Thw hardware is not the issue. It's the model architectures leading to cascading errors
  - matwood 10mo ago
    Are you saying that Google, Meta, Amazon, etc... can't do software? It's the bread and butter of these companies. The CUDA moat is important to hold off the likes of AMD, but hardware like TPUs for internal use or other big software makers is not a big hurdle. Of course Huang will lean on the software being key because he sees the hardware competition catching up.
    - sigmoid10 10mo ago
      Huang said that many years ago, long before ChatGPT or the current AI hype were a thing. In that interview he said that their costs for software R&D and support are equal or even bigger than their hardware side. They've also been hiring top SWE talent for almost two decades now. None of the other companies have spent even close to this much time and money on GPU software, at least until LLMs became insanely popular. So I'd be surprised to see them catch up anytime soon.
- Glemkloksdjf 10mo ago
  Nvidia has everything they need to build the most advanced GPU Chip in the world and mass produce it. Everything. They can easily just do this for more optimized Chips. "easily" in sense of that wouldn't require that much investment. Nvidia knows how to invest and has done this for a long time. Their Ominiverse or robots platform isaac are all epxensive. Nvidia has 10x more software engineers than AMD
  - farseer 10mo ago
    They still go to TSMC for fab, and so does everyone else.
    - Glemkloksdjf 10mo ago
      For sure. But they also have high volumne and know how to do everything. Also certain companies normally don't like to do things themselves if they don't have to. Nonetheless nvidia is were it is because it has cude and an ecoysystem. Everyone uses this ecosystem and then you just run that stuff on the bigger version of the same ecosystem.
randomNumber7 10mo ago
If you look at the history how GPUs evolved: 1. there had be fixed function hardware for certain graphics stages 2. Programmable massively parallel hardware took over. Nvidia was at the forefront of this. TPUs seem to me similar to fixed function hardware. For Nvidia it's a step backwards and even though they go into this direction recently I can't see them go all the way. Otherwise you don't need cuda, but hardware guy's that write verilog or vhdl. They don't have that much of an edge there.
captainbland 10mo ago
Nvidia is already in the TPU race aren't they? This is exactly what the tensor cores on their current products are supposed to do, but they're just more heterogeneous GPU based architectures and exist with CUDA cores etc. on the same die. I think it should be within their capability to make a device which devotes an even higher ratio of transistors to tensor processing.
herbst 10mo ago
Why dig for gold when you are the gold standard for the shovel already?

by binkHN 10mo ago

sysguest 10mo ago

$5 trillion valuation doesn't mean it has $5 trillion cash in pocket -- so "it depends"

dragonwriter 10mo ago

> What's to stop them from entering the TPU race with their $5 trillion valuation? Valuation isn’t available money; they'd have to raise more money in the current, probably tighter for them, investment environment to enter the TPU race, since the money they have already raised that that valuation is based on is already needed to provide runway for what they are already doing without putting money into the TPU race

matwood 10mo ago

As others mentioned, 5T isn't money available to NVDA. It could leverage that to buy a TPU company in an all stock deal though. The bigger issue is that entering a 'race' implies a race to the bottom. I've noted this before, but one of NVDA's biggest risks is that its primary customers are also technical, also make hardware, also have money, and clearly see NVDA's margin (70% gross!!, 50%+ profit) as something they want to eliminate. Google was first to get there (not a surprise), but Meta is also working on its own hardware along with Amazon. This isn't a doom post for NVDA the company, but its stock price is riding a knifes edge. Any margin or growth contraction will not be a good day for their stock or the S&P.

sigmoid10 10mo ago
Making the hardware is actually the easy part. Everyone and their uncle who had some cash have tried by now: Microsoft, Meta, Tesla, Huawei, Amazon, Intel - the list goes on and on. But Nvidia is not a chip company. Huang himself said they are mostly a software company. And that is how they were able to build a gigantic moat. Because noone else has even come close on the software side. Google is the only one who has had some success on this side, because they also spent tons of money and time on software refinement by now, while all the other chips vanished into obscurity.
- sanjayjc 10mo ago
  Genuine question: given LLMs' inexorable commoditization of software, how soon before NVDA's CUDA moat is breached too? Is CUDA somehow fundamentally different from other kinds of software or firmware?
  - tomrod 10mo ago
    Current Gen LLMs are not breaching the moat yet.
    - fzzzy 10mo ago
      Yeah they are. llama.cpp has had good performance on cpu, amd, and apple metal for at least a year now.
      - tomrod 10mo ago
        Thw hardware is not the issue. It's the model architectures leading to cascading errors
- matwood 10mo ago
  Are you saying that Google, Meta, Amazon, etc... can't do software? It's the bread and butter of these companies. The CUDA moat is important to hold off the likes of AMD, but hardware like TPUs for internal use or other big software makers is not a big hurdle. Of course Huang will lean on the software being key because he sees the hardware competition catching up.
  - sigmoid10 10mo ago
    Huang said that many years ago, long before ChatGPT or the current AI hype were a thing. In that interview he said that their costs for software R&D and support are equal or even bigger than their hardware side. They've also been hiring top SWE talent for almost two decades now. None of the other companies have spent even close to this much time and money on GPU software, at least until LLMs became insanely popular. So I'd be surprised to see them catch up anytime soon.
Glemkloksdjf 10mo ago
Nvidia has everything they need to build the most advanced GPU Chip in the world and mass produce it. Everything. They can easily just do this for more optimized Chips. "easily" in sense of that wouldn't require that much investment. Nvidia knows how to invest and has done this for a long time. Their Ominiverse or robots platform isaac are all epxensive. Nvidia has 10x more software engineers than AMD
- farseer 10mo ago
  They still go to TSMC for fab, and so does everyone else.
  - Glemkloksdjf 10mo ago
    For sure. But they also have high volumne and know how to do everything. Also certain companies normally don't like to do things themselves if they don't have to. Nonetheless nvidia is were it is because it has cude and an ecoysystem. Everyone uses this ecosystem and then you just run that stuff on the bigger version of the same ecosystem.

randomNumber7 10mo ago

If you look at the history how GPUs evolved: 1. there had be fixed function hardware for certain graphics stages 2. Programmable massively parallel hardware took over. Nvidia was at the forefront of this. TPUs seem to me similar to fixed function hardware. For Nvidia it's a step backwards and even though they go into this direction recently I can't see them go all the way. Otherwise you don't need cuda, but hardware guy's that write verilog or vhdl. They don't have that much of an edge there.

captainbland 10mo ago

Nvidia is already in the TPU race aren't they? This is exactly what the tensor cores on their current products are supposed to do, but they're just more heterogeneous GPU based architectures and exist with CUDA cores etc. on the same die. I think it should be within their capability to make a device which devotes an even higher ratio of transistors to tensor processing.

herbst 10mo ago

Why dig for gold when you are the gold standard for the shovel already?