I watched Huang's Computex presentation and was pretty impressed by the hardware they're launching, but it's worth noting some caveats. While it was announced t

I watched Huang's Computex presentation and was pretty impressed by the hardware they're launching, but it's worth noting some caveats. While it was announced to be in "full production," according to the blog post, DGX GH200 won't be available until the end of the year, which puts it about on the same timeline as AMD's delivery of MI300. Also, while the 1 exaFLOPS topline number is impressive, this has some asterisks. Each GH H100 GPU only does 34 TFLOPS of FP64 according to the data sheet. [1] At 256 nodes, this is a mere 8.7 petaFLOPS, or 0.0087 exaFLOPS. You only get to the 1 exaFLOPS number (from looking at the data sheet) if you are doing sparse FP8 Tensor Core FLOPS (3968 TFLOPS/GH, non-sparse is halved). It's worth keeping this in mind when comparing to something like Frontier (1.1 exaFLOPS) [2] or the upcoming El Capitan (expected 2 exaFLOPS) [3] - Top500 uses LINPACK, which benchmarks FP64 FLOPS. Of course, for AI training, FP8 or BF16 is probably the most relevant numbers for perf/W and perf/$... Frontier and El Capitan are each estimated to cost ~$600M, and while exact numbers weren't given, I'd expect a full 256-node DGX H200 to come in between $50-100M. AMD will be having a "Data Center and AI" event on June 13th, so we'll get to see soon how competitive they are (the announced MI300 specs is 24 x Zen 4 cores with a CDNA3 architecture that is 8X faster than MI250X (383 FP16/BF16 TFLOPS), so about 3000 TFLOPS, with 128GB of unified HBM3 on a 8196 bit bus (6.5TB/s theoretical), which is in the same ballpark as Grace Hopper - I think it'll mostly come down to software, but with drop-in PyTorch support, OpenAI's Triton, etc, I'm somewhat optimistic that it will be worth it for big players do to some software lift (that others can benefit from), if the cost competitiveness of the hardware is there. For details already made public, see: https://www.tomshardware.com/news/new-amd-instinct-mi300-details-emerge-debuts-in-2-exaflop-el-capitan-supercomputer https://www.tomshardware.com/news/new-amd-instinct-mi300-det... [1] https://resources.nvidia.com/en-us-dgx-gh200/nvidia-grace-hopper-superchip-datasheet https://resources.nvidia.com/en-us-dgx-gh200/nvidia-grace-ho... [2] https://en.wikipedia.org/wiki/Frontier_(supercomputer) https://en.wikipedia.org/wiki/Frontier_(supercomputer) [3] https://en.wikipedia.org/wiki/El_Capitan_(supercomputer) https://en.wikipedia.org/wiki/El_Capitan_(supercomputer)