3 ms·
One H100 has the same memory bandwidth as the two CPUs in each of the compute nodes. But the supercomputer has >9000 compute nodes, so 62x H100 GPUs might fall
by AnthonyMouse 3y ago
One H100 has the same memory bandwidth as the two CPUs in each of the compute nodes. But the supercomputer has >9000 compute nodes, so 62x H100 GPUs might fall short there.
They also might not be measuring the same thing. H100 GPUs have "3026 TFLOPS":
https://www.pny.com/nvidia-h100 https://www.pny.com/nvidia-h100
...at FP8. 51 TFLOPS at FP32. Supercomputer lists typically use FP64.