4 ms·
Google is using them in prod. I think they're so hungry for chips internally that cloud isn't getting much support in selling them.
by hiddencost 3y ago
Google is using them in prod. I think they're so hungry for chips internally that cloud isn't getting much support in selling them.
- amelius 3y agoOr maybe they are just using nVidia. Who knows ...
- eklitzke 3y agoLots of people know.
- nivekney 3y agomarks as solved
- vineyardmike 3y agoBeyond the fact that this is hardly a secret, there’s lots of other signs. 1. They have bought far less from NVidia than other hyper scalers, and they literally can’t vomit without saying “AI”. They have to be running those models on something. They have purchased huge amounts of chips from fabs, and what else would that be? 2. They have said they use them. Should be pretty obvious here. 3. They maintain a whole software stack for them, they design the chips, etc. Then they don’t really try to sell the TPU. Why else would they do this?
- jeffbee 3y agoThey have announced publicly using TPUs for inference, as far back as 2016. They did not offer TPUs for Cloud customers until 2017. The development is clearly driven by internal use cases. One of the features they publicly disclosed as TPU-based was Smart Reply and that launched in 2015. So their internal use of TPUs for inference goes back nearly a decade.
- danjl 3y agoI would guess that Google's vertexAI managed solution uses TPUs. Also Google uses them internally to train and infer for all their research products.
- sciencesama 3y ago80 to 90% are consumed internally !! Only from version 5 it is planned to be customer focussed !!
- michaelt 3y agoWhile you can use TPUs with vertexai, it's just virtual machines - you can have one with an nvidia card if you like.
- VirusNewbie 3y agoThey're getting swallowed up by Anthropic and the other huge spenders: https://www.prnewswire.com/news-releases/google-announces-expansion-of-ai-partnership-with-anthropic-301981815.html https://www.prnewswire.com/news-releases/google-announces-ex... "Partnership includes important new collaborations on AI safety standards, committing to the highest standards of AI security, and use of TPU v5e accelerators for AI inference "
- _b 3y agoI think this is right, in part because I've been told exactly this from people who work for Google and their job is to sell me cloud stuff- i.e., they say they have so much internal demand they aren't pushing TPUs for external use. Hence external pricing and support just isn't that great right now. But presumably when capacity catches up they'll start pushing TPUs again.
- natbobc 3y agoFeels like a bad point in the curve to try and sell them. “Oh our internal hypecycle is done… we’ll put them in the market now that they’re all worn out.
- rj45jackattack 3y agoSounds like ButterflyLabs.