3 ms·
Nvidia GPUs over significantly more performance per card than Intel and AMD. In general the limiting factor is total computation, so people will pay a premium
by hx8 2y ago
Nvidia GPUs over significantly more performance per card than Intel and AMD. In general the limiting factor is total computation, so people will pay a premium for the best. I'm not familiar with Google's hardware, and don't think it's generally available.
- ArtTimeInvestor 2y agoWhat about AMD's MI300X? I see reports all over the web that it runs LLMs at a similar speed as an H100.
- season2episode3 2y agoIt’s available on GCP through Cloud TPU, though Nvidia on GCP is still a far bigger business.
- ArtTimeInvestor 2y agoCan you run LLAMA on Google's TPUs?
- xnx 2y agoYes. You can serve Llama on TPU: https://cloud.google.com/vertex-ai/docs/predictions/serve-llama3-with-saxml-tpu https://cloud.google.com/vertex-ai/docs/predictions/serve-ll... You can also train Llama on TPU: https://cloud.google.com/tpu/docs/v5p-training https://cloud.google.com/tpu/docs/v5p-training