29 ms·
TPUs are slow but efficient and AMD has bugs but for some things works quite well. Nvidia is obviously the gold standard.
by CuriouslyC 2y ago
TPUs are slow but efficient and AMD has bugs but for some things works quite well. Nvidia is obviously the gold standard.
- felarof 2y agoHaha, TPUs are not slow :) All of Google's training (including Gemini models) is done on TPUs. There are good 1p [a] and 3p [b] benchmarks comparing TPUs vs NVIDIA GPUs. [a] - https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/community-content/vertex_model_garden/benchmarking_reports/jax_vit_benchmarking_report.md https://github.com/GoogleCloudPlatform/vertex-ai-samples/blo... [b] - https://arxiv.org/pdf/2309.07181 https://arxiv.org/pdf/2309.07181