Back of the envelope math is that H100 is twice as fast as A100 (task may vary). So your 1000-node A100 very, very fast. Now, the GPU-to-GPU links (NVLink) mig

Back of the envelope math is that H100 is twice as fast as A100 (task may vary). So your 1000-node A100 very, very fast. Now, the GPU-to-GPU links (NVLink) might often give them a big advantage for some workloads, letting them exchange data without going through the CPU, and virtually address more memory if your want to manipulate very large models. So it's hard to answer properly without knowing the topology of your cluster. Also, note that this "supercomputer", is probably "just" a DGX H100 in Google's DC.