I think hardware acquisition costs can dominate GPU prices, so perf/watt might not be a realistic measure for the cost of running such large scale/high performa

I think hardware acquisition costs can dominate GPU prices, so perf/watt might not be a realistic measure for the cost of running such large scale/high performance experiments. For instance, the DGX-2 has an MSRP of $399,000 and consumes 10 kW of power [1]. The average commercial electricity cost across the US is about $0.11/kWh [2], so a DGX-2 running at full tilt costs $1.10 an hour in electricity. Thus, a DGX-2 running at 100% utilization for 3 years costs $9,636 in electricity, which is ~2.5% of the cost of the box itself. Of course, you probably could get DGX-2s for a lot cheaper if you are buying 100 of them, but the acquisition costs are still going to be significant vis-a-vis power costs. [1]: https://www.anandtech.com/show/12587/nvidias-dgx2-sixteen-v100-gpus-30-tb-of-nvme-only-400k https://www.anandtech.com/show/12587/nvidias-dgx2-sixteen-v1... [2]: https://www.pacificpower.net/about/rr/cpc.html https://www.pacificpower.net/about/rr/cpc.html