Haha, TPUs are not slow :) All of Google's training (including Gemini models) is done on TPUs. There are good 1p [a] and 3p [b] benchmarks comparing TPUs vs NV

Haha, TPUs are not slow :) All of Google's training (including Gemini models) is done on TPUs. There are good 1p [a] and 3p [b] benchmarks comparing TPUs vs NVIDIA GPUs. [a] - https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/community-content/vertex_model_garden/benchmarking_reports/jax_vit_benchmarking_report.md https://github.com/GoogleCloudPlatform/vertex-ai-samples/blo... [b] - https://arxiv.org/pdf/2309.07181 https://arxiv.org/pdf/2309.07181