4 ms·

Why is that a huge advantage over AI competitors? Just not having to fight for limited Nvidia supply?

aseipp 2y ago
That is one factor, but another is total cost of ownership. At large scales something that's 1/2 the overall speed but 1/3rd the total cost is still a net win by a large margin. This is one of the reasons why every major hyperscaler is, to some extent, developing their own hardware e.g. Meta, who famously have an insane amount of Nvidia GPUs. Of course this does not mean their models will necessarily be proportionally better, nor does it mean Google won't buy GPUs for other reasons (like providing them to customers on Google Cloud.)
xnx 2y ago
Yes. And cheaper operating cost per TFLOP.
badlucklottery 2y ago
Vertical integration. Nvidia is making big bucks "selling shovels in a gold rush". Google has made their own shovel factory and they can avoid paying Nvidia's margins.
bufferoverflow 2y ago
TPUs are cheaper and faster than GPUs. But it's custom silicon. Which means barrier to entry is very very high.
- felarof 2y ago
  > Which means barrier to entry is very very high. +1 on this. The tooling to use TPUs still needs more work. But we are betting on building this tooling and unlocking these ASIC chips (https://github.com/felafax/felafax https://github.com/felafax/felafax).
- ein0p 2y ago
  They are cheaper, yes. But GPUs are faster and easier to program.
- imtringued 2y ago
  Nah, just take a look at their own comparison page: https://cloud.google.com/tpu/docs/v6e https://cloud.google.com/tpu/docs/v6e The A100 (from 2020) had 300 TFLOPs (1.5x) and 80GB HBM. It's only now with Trillium that they are starting to actually beat Nvidia in a chip vs chip battle, but being faster per chip was never the point. TPUs were supposed to be mass produced and connected into pods so that they become cheaper and they indeed are.
  - hustwindmaple1 2y ago
    Exactly. Kind of like the old days when they put together a massive amount of commodity CPUs to build search.

That is one factor, but another is total cost of ownership. At large scales something that's 1/2 the overall speed but 1/3rd the total cost is still a net win by a large margin. This is one of the reasons why every major hyperscaler is, to some extent, developing their own hardware e.g. Meta, who famously have an insane amount of Nvidia GPUs. Of course this does not mean their models will necessarily be proportionally better, nor does it mean Google won't buy GPUs for other reasons (like providing them to customers on Google Cloud.)

xnx 2y ago

Yes. And cheaper operating cost per TFLOP.

badlucklottery 2y ago

Vertical integration. Nvidia is making big bucks "selling shovels in a gold rush". Google has made their own shovel factory and they can avoid paying Nvidia's margins.

bufferoverflow 2y ago

TPUs are cheaper and faster than GPUs. But it's custom silicon. Which means barrier to entry is very very high.

felarof 2y ago
> Which means barrier to entry is very very high. +1 on this. The tooling to use TPUs still needs more work. But we are betting on building this tooling and unlocking these ASIC chips (https://github.com/felafax/felafax https://github.com/felafax/felafax).
ein0p 2y ago
They are cheaper, yes. But GPUs are faster and easier to program.
imtringued 2y ago
Nah, just take a look at their own comparison page: https://cloud.google.com/tpu/docs/v6e https://cloud.google.com/tpu/docs/v6e The A100 (from 2020) had 300 TFLOPs (1.5x) and 80GB HBM. It's only now with Trillium that they are starting to actually beat Nvidia in a chip vs chip battle, but being faster per chip was never the point. TPUs were supposed to be mass produced and connected into pods so that they become cheaper and they indeed are.
- hustwindmaple1 2y ago
  Exactly. Kind of like the old days when they put together a massive amount of commodity CPUs to build search.