3 ms·
Gemini doesn't run on GPUs, it runs on Google's custom TPU silicon. The TPU v4 was already competitive with Nvidia, and the new v5p is 2.4 times faster and mor
by panarky 3y ago
Gemini doesn't run on GPUs, it runs on Google's custom TPU silicon.
The TPU v4 was already competitive with Nvidia, and the new v5p is 2.4 times faster and more power efficient than v4.
- minimaxir 3y agoThat may be true: the economics of TPUs are more complicated but they certainly aren't as much in demand.
- echelon 3y ago> aren't as much in demand. Google gets to manufacture that demand at scale.
- TwentyPosts 3y agoThat's primarily just because Google is doing a poor job offering them to people. Bad documentation, and few trust Google enough to want to tie themselves to Google's trusted hardware. Also, Google is using all these TPUs for their own training and inference at the moment.
- saberience 3y agoDo you have a solid source for this claim? Everything I have read/heard tells me that companies would much, much rather have access to Nvidia chips than Googles TPUs.
- pkaye 3y agoI think there are different size Gemini models and some of them can fit on a phone with TPUs or similar and since each user will use their own phone, it offloads the computation.
- mupuff1234 3y agoBecause TPUs are aimed at Google workloads so probably not as easy to work with outside of Google.
- dagmx 3y agoNVIDIA chips are better for training due to the sheer prevalence of CUDA and NVIDIA being ahead of the curve in providing data centre hardware to scale up. They’re also not keeping stuff for just themselves. When it comes to inference, most of that isn’t really relevant. TPUs or any other specialized hardware will have a higher ROI. But Google aren’t giving them to others either I know you’re asking for a link, but it’s not really something other than Google can prove. However it is a reasonable extrapolation for anyone familiar with the product space to make.