5 ms·

Googler here, if you haven’t looked at TPUs in a while check out the v5. They support PyTorch/JAX now, makes them much easier to use than TF only.

tw04 3y ago
Where can I buy a TPU v5 to install in my server? If the answer is “cloud”: that’s why NVidia is wiping the floor.
- deleted 3y ago
  [deleted]
- ShamelessC 3y ago
  You probably can't even rent them from Google if you wanted to, in my experience.
  - inhumantsar 3y ago
    https://cloud.google.com/tpu https://cloud.google.com/tpu
    - jedberg 3y ago
      I think OPs point was Google claims to have TPUs in their cloud but in reality they are rarely available.
- foota 3y ago
  How many people are out there buying H100s for their personal use?
  - Workaccount2 3y ago
    Probably many orders of magnitude greater than those buying TPU's for personal use...
    - foota 3y ago
      Technically correct, but only because TPUs aren't for sale. H100s cost like 30,000 USD, if you can even get one.
      - immibis 3y ago
        So in other words, every AI company has at least 20.
  - michaelt 3y ago
    Ah, but part of the reason for CUDA's success is that the open source developer who wants to run unit tests or profile their kernel can pick up a $200 card. That PhD student with a $2000 budget can pick up a card. Academic lab with $20,000 for a beefy server, or tiny cluster? nvidia will take their money. And that's all fixed capital expenditure - there's no risk a code bug or typo by an inexperienced student will lead to a huge bill. Also, if you're looking for an alternative to CUDA because you dislike vendor lock-in, switching to something only available in GCP would be an absurd choice.
    - _ea1k 3y ago
      I'm really shocked at how dependent companies have become on the cloud offerings. Want a GPU? Those are expensive, lets just rent on Amazon and then complain about operational costs! I've noticed this at companies. Yeah, the cloud is expensive, but you have a data center, and a few servers with RTX 3090s aren't expensive. A lot of research workloads can run on simple, cheap hardware. Even older Nvidia P40s are still useful.
  - rhdunn 3y ago
    Probably not many. However, 4090s would be a different situation. There are plenty of guides on running LLMs, stable diffusion, etc. on local hardware. The H100s would be for businesses looking to get into this space.

tw04 3y ago

Where can I buy a TPU v5 to install in my server? If the answer is “cloud”: that’s why NVidia is wiping the floor.

deleted 3y ago
[deleted]
ShamelessC 3y ago
You probably can't even rent them from Google if you wanted to, in my experience.
- inhumantsar 3y ago
  https://cloud.google.com/tpu https://cloud.google.com/tpu
  - jedberg 3y ago
    I think OPs point was Google claims to have TPUs in their cloud but in reality they are rarely available.
foota 3y ago
How many people are out there buying H100s for their personal use?
- Workaccount2 3y ago
  Probably many orders of magnitude greater than those buying TPU's for personal use...
  - foota 3y ago
    Technically correct, but only because TPUs aren't for sale. H100s cost like 30,000 USD, if you can even get one.
    - immibis 3y ago
      So in other words, every AI company has at least 20.
- michaelt 3y ago
  Ah, but part of the reason for CUDA's success is that the open source developer who wants to run unit tests or profile their kernel can pick up a $200 card. That PhD student with a $2000 budget can pick up a card. Academic lab with $20,000 for a beefy server, or tiny cluster? nvidia will take their money. And that's all fixed capital expenditure - there's no risk a code bug or typo by an inexperienced student will lead to a huge bill. Also, if you're looking for an alternative to CUDA because you dislike vendor lock-in, switching to something only available in GCP would be an absurd choice.
  - _ea1k 3y ago
    I'm really shocked at how dependent companies have become on the cloud offerings. Want a GPU? Those are expensive, lets just rent on Amazon and then complain about operational costs! I've noticed this at companies. Yeah, the cloud is expensive, but you have a data center, and a few servers with RTX 3090s aren't expensive. A lot of research workloads can run on simple, cheap hardware. Even older Nvidia P40s are still useful.
- rhdunn 3y ago
  Probably not many. However, 4090s would be a different situation. There are plenty of guides on running LLMs, stable diffusion, etc. on local hardware. The H100s would be for businesses looking to get into this space.