4 ms·
I was curious about energy efficiency and took two samples from the linked MLPerf GPT3 results. H100 seems about three times more efficient than Gaudi2. 256
by akreal 3y ago
I was curious about energy efficiency and took two samples from the linked MLPerf GPT3 results. H100 seems about three times more efficient than Gaudi2.
256 Gaudi2 600W TDP: 256 * (442.578 / 60) * 0.6 = ~1133 kWh
512 H100 700W TDP: 512 * (64.264 / 60) * 0.7 = ~384 kWh
- sm_1024 3y agoI'm guessing H100 has 2x host energy overhead for connecting those GPUs? That might offset some of the perf/W benefits of nvidia's offering.