3 ms·
> why wouldn't Google be selling shovels They do sell them - but through their struggling cloud business. Either way, Nvidia's margin is google's opportunity t
by vineyardmike 2y ago
> why wouldn't Google be selling shovels
They do sell them - but through their struggling cloud business. Either way, Nvidia's margin is google's opportunity to lower costs.
> I can see Google's custom chips are 15x to 30x slower to train AI
TPUs are designed for inference not training - they're betting that they can serve models to the world at a lower cost structure than their competition. The compute required for inference to serve their billions of customers is far greater than training costs for models - even LLMs. They've been running model inference as a part of production traffic for years.
- refulgentis 2y agoThis breaks my brain, because I know Google trains it models on TPUs and they're seen as faster, and if they're better at inference, and can train, then why is Nvidia in a unique position? My understanding was always it's as simple as it required esoteric tooling
- smueller1234 2y agoMultiple types of TPUs. (I work for Google, but the above is public information.)
- vineyardmike 2y agoBecause people generally don’t use TPUs outside of Google. The tooling is different, the access is metered through GCP, etc. Nvidia is in a vaguely unique position in that their products have great tooling support and few companies sell silicon at their scale.
- refulgentis 2y agoCorrect, I'm pointing out politely that's in conflict with the person I'm replying to.
- koe123 2y agoPossibly naive but I very much view CUDA and its integration into ML frameworks being nvidias moat
- uluyol 2y agoGoogle most certainly uses TPUs for training.