3 ms·
> Investing that much in hardware that depreciates over 5+ years and is theoretically still usable at the end, maybe Isn’t that exactly what’s happening? A $3
by throwup238 2y ago
> Investing that much in hardware that depreciates over 5+ years and is theoretically still usable at the end, maybe
Isn’t that exactly what’s happening?
A $300k 8x H100 pod with 5kW power supply burns at most $6k per year in power at $0.15/kWh. The majority of the money is going to capital equipment for the first time in the software industry in decades.
These top of the line chips last for much longer in the depreciation game. The A100 was released in 2020 but cloud providers still have trouble meeting demand and charge a premium for them.
- dagmx 2y agoHow are you getting 5kW? NVIDIA claim 10.2kW for a DGX H100 pod. https://docs.nvidia.com/dgx/dgxh100-user-guide/introduction-to-dgxh100.html#:~:text=130.45%20kg)%20max-,Power%20Specifications,distribution%20of%20the%20power%20load.&text=10.2%20kW%20max https://docs.nvidia.com/dgx/dgxh100-user-guide/introduction-.... Your point still stands though where power is a fraction of the cost. The bigger issue is power + cooling and how many units are needed to train the better models.
- throwup238 2y agoServes me right for doing back of napkin math instead of just looking it up. The SXM4 A100s I use have a TDP of 400W and I hadn't realized the higher power H100 are up to 800W.
- deleted 2y ago[deleted]