TPU v5p is ~2 times slower than H100 at larg(ish)-scale training (order of 10k chips) [1]. And they already have v6 [2]. I think it's safe to say that they are

TPU v5p is ~2 times slower than H100 at larg(ish)-scale training (order of 10k chips) [1]. And they already have v6 [2]. I think it's safe to say that they are fairly close to Nvidia in terms of performance. [1] https://mlcommons.org/benchmarks/training/ https://mlcommons.org/benchmarks/training/ [2] https://cloud.google.com/blog/products/compute/introducing-trillium-6th-gen-tpus https://cloud.google.com/blog/products/compute/introducing-t...