I'm not so sure there. The factor of 72 is accidentially also the number of GPUs in a full GB200 DGX rack[0]. The phrasing "and it reaches 72,000 TPS/server at

I'm not so sure there. The factor of 72 is accidentially also the number of GPUs in a full GB200 DGX rack[0]. The phrasing "and it reaches 72,000 TPS/server at our highest throughput configuration" also hints at something being fishy here. They carefully use the phrases "node" earlier, and "server" later, without getting specific by what they mean a "server" to be. Also, for that 72000 figure, there is no mention of batching at all. The very short article [2] linked in [0] which is supposed to be the independent source of those numbers also doesn't specify any details to that effect. In general, I've learned to treat Nvidia numbers very carefully. They are well-known for misrepresenting apples-to-orange-elefants figures such as comparing FP16, FP8 or FP4 FLOPS, thereby grossly overstating the performance advantages of their new architectures[3]. [0] https://developer.nvidia.com/blog/blackwell-breaks-the-1000-tps-user-barrier-with-metas-llama-4-maverick/ https://developer.nvidia.com/blog/blackwell-breaks-the-1000-... [1] > NVIDIA DGX™ GB200 is purpose-built for training and inferencing trillion-parameter generative AI models. Designed as a rack-scale solution, each liquid-cooled rack features 36 NVIDIA GB200 Grace Blackwell Superchips—–36 NVIDIA Grace CPUs and 72 Blackwell GPUs https://www.nvidia.com/en-eu/data-center/dgx-gb200/ https://www.nvidia.com/en-eu/data-center/dgx-gb200/ [2] https://www.linkedin.com/feed/update/urn:li:activity:7331470006850883584/ https://www.linkedin.com/feed/update/urn:li:activity:7331470... [3] https://dev.to/maximsaplin/nvidias-1000x-performance-boost-claim-verified-j7f https://dev.to/maximsaplin/nvidias-1000x-performance-boost-c...