29 ms·
The 10x figure is made up. We don't know if or by how much the TPU ecosystem is cheaper than Nvidia. Firstly, the h100 margin number appears to be looking pure
by mike_hearn 2y ago
The 10x figure is made up. We don't know if or by how much the TPU ecosystem is cheaper than Nvidia.
Firstly, the h100 margin number appears to be looking purely at the manufacturing cost vs sale price. Where are amortised driver, software stack and design costs in that? Nvidia has won because of the strength of the CUDA stack, including their uniform driver codebase which isn't free to develop.
The cost of a tpu or GPU isn't simply in the hardware costs. Google has to maintain an entire parallel software ecosystem as well. Then we can assume Nvidia is spreading design costs over many clients including gamers and the console industry as well, whereas Google isn't.
It might well still be a saving for Google because Nvidia is making bank right now, but if it was actually easy to replace these chips with in house designs and software stacks then all the big cloud companies would be doing it. They're trying, but so far those projects haven't worked out.