4 ms·
> I was more contextualizing this, where a potential 75% increase in power (on a smaller node) to deliver 50% more performance is less impressive: That's misle
by rajnathani 4y ago
> I was more contextualizing this, where a potential 75% increase in power (on a smaller node) to deliver 50% more performance is less impressive:
That's misleading, as the GPU power utilization during ResNet is very likely not 100% for either the A100 or H100.
- lunixbochs 4y agoI'm able to trivially sustain 83% (250W / 300W) of the TDP of an Ampere A40 training a model the size of resnet50 with a basic pytorch training loop. Remember the lower-TDP PCIe H100 has 20% slower compute and 33% slower memory than the SXM model, suggesting the increased power delivery from the PCIe (350W) to SXM (700W) model is a major factor in the performance even for H100 vs H100. I don't think it's misleading to say that power is extremely likely to be a factor in the demonstrated performance increase from A100 to H100 for non-transformer workloads until proven otherwise. I don't think anyone here has a DGX H100 in hand yet to test this. Edit: wait, also I looked closer at the numbers - in MLPerf 2.1, NVIDIA only submitted results for 1x H100. There's no 1x A100 result submitted for resnet50, just an 8x A100 number, which NVIDIA seems to have divided by 8 to get a "per accelerator" number to compare to their 1x H100. That doesn't feel like a clean comparison, as you can have performance loss when scaling. I'd rather see 1v1 or 8v8.