3 ms·

> Google can sell VMs with the tensor chips attached, like GPUs. They have already been doing this for quite a while now and even when offered free via TRC bar

> Google can sell VMs with the tensor chips attached, like GPUs. They have already been doing this for quite a while now and even when offered free via TRC barely anyone uses TPUs. There is nothing to suggest that Google as an organization is shifting focus to be the HPC cloud provider for the world. As it stands TPU cloud access really seems ancillary to their own internal needs. > If Apple/MSFT/AWS/Goog/Meta start serious building their own chips, NVidia could be left out of the top end. That's a big "if", especially within two years, given that this chip design/manufacturing isn't really a core business interest for any of those companies (other than Google which has massive internal need and potentially Apple who have never indicated interest in being a cloud provider). They certainly could compete with Nvidia for the top-end, but it would be really hard and how much would the vertical integration actually benefit their bottom line? A 2048 GPU SuperPOD is what, like 30M? There's also the risk that the not-always-friendly DoJ gets anti-trusty if a cloud provider has a massive advantage and is locking the HW in their walled garden.

fragmede 3y ago
> barely anyone uses TPUs What are you basing that on? I'm not aware of GCP having released any numbers on their usage.
- haldujai 3y ago
  Anecdotal data warning but for context my research is in medical informatics and I've quite extensively followed publications on transformers dating back to the early BERT variants (including non-medical). I'm making that statement as my experience (easily several hundreds of publications read or reviewed over 3 years) is that it is very uncommon to see TPU's mentioned or TRC acknowledged in any non-Google transformer paper (especially major publications) dating back to the early BERT family of models despite the fact that Google is very generous with research credits (they'll give out preemptible v3-32s and v3-64s for 14 days with little question, presumably upgraded now as I haven't asked for credits in a while). Fully acknowledge this isn't quality evidence to back my claim and I'm happy to be proven wrong but I'm very confident a literature review would support this as when I tried to use TPUs myself I couldn't find much. This doesn't account for industry use, there is probably a non-insignificant amount of enterprise customers still using AutoML (I can think of a few at least) which I believe uses the TPU cloud but I would be surprised if many use TPU nodes directly outside of Jax shops like cohere and anyone still using TF. PyTorch XLA has just breaks too much otherwise and when I last tried to use it in January of this year there was still quite a significant throughput reduction on TPUs. Additionally when using nodes there is a steeper learning curve on the ops side (VM, storage, Stackdriver logging) that make working with them harder than spinning up a A100x8 which is relatively cheap, cheaper than the GCP learning curve for sure.
  - sangnoir 3y ago
    > Anecdotal data warning but for context my research is in medical informatics Isn't Medical Informatics inherently biased against the cloud? That's my uninformed guess as an outsider.
bagacrap 3y ago
It seems like one of these two things must be true: A) Nvidia's TAM is not really what the stock is priced foe B) Google will try to enter this market and compete Either way NVDA looks perilously pricey, not that that is very predictive of anything (see TSLA).

fragmede 3y ago

> barely anyone uses TPUs What are you basing that on? I'm not aware of GCP having released any numbers on their usage.

haldujai 3y ago
Anecdotal data warning but for context my research is in medical informatics and I've quite extensively followed publications on transformers dating back to the early BERT variants (including non-medical). I'm making that statement as my experience (easily several hundreds of publications read or reviewed over 3 years) is that it is very uncommon to see TPU's mentioned or TRC acknowledged in any non-Google transformer paper (especially major publications) dating back to the early BERT family of models despite the fact that Google is very generous with research credits (they'll give out preemptible v3-32s and v3-64s for 14 days with little question, presumably upgraded now as I haven't asked for credits in a while). Fully acknowledge this isn't quality evidence to back my claim and I'm happy to be proven wrong but I'm very confident a literature review would support this as when I tried to use TPUs myself I couldn't find much. This doesn't account for industry use, there is probably a non-insignificant amount of enterprise customers still using AutoML (I can think of a few at least) which I believe uses the TPU cloud but I would be surprised if many use TPU nodes directly outside of Jax shops like cohere and anyone still using TF. PyTorch XLA has just breaks too much otherwise and when I last tried to use it in January of this year there was still quite a significant throughput reduction on TPUs. Additionally when using nodes there is a steeper learning curve on the ops side (VM, storage, Stackdriver logging) that make working with them harder than spinning up a A100x8 which is relatively cheap, cheaper than the GCP learning curve for sure.
- sangnoir 3y ago
  > Anecdotal data warning but for context my research is in medical informatics Isn't Medical Informatics inherently biased against the cloud? That's my uninformed guess as an outsider.

bagacrap 3y ago

It seems like one of these two things must be true: A) Nvidia's TAM is not really what the stock is priced foe B) Google will try to enter this market and compete Either way NVDA looks perilously pricey, not that that is very predictive of anything (see TSLA).