3 ms·
Energy is like 10-20% of the cost of AI. The rest is mostly hardware depreciation.
by onlyrealcuzzo 5mo ago
Energy is like 10-20% of the cost of AI.
The rest is mostly hardware depreciation.
- stingraycharles 5mo agoSo you think they’re running the same types of state of the art Nvidia deployments?
- onlyrealcuzzo 5mo agoIt's supposed to be even MORE expensive: Nvidia H100: Typically priced around $25,000–$30,000 (global MSRP). Huawei Ascend 910C: Reported to cost roughly $28,000, yet it delivers only 60% of the inference performance of the Nvidia H100. Google's TPUs are significantly cheaper for Google for inference. That's pretty much it. There's a reason nVidia has an 80% margin right now.
- thrownthatway 5mo agoMSRP is irrelevant in this context.
- cmiles8 5mo agoCorrect. There are challenges getting enough energy to new data center builds but the cost of the energy is low relative to other costs.