4 ms·

Last point should also be for llm. Not just crypto

fc417fc802 3mo ago
Not true. A B200 "only" draws 1 kW. At 16¢ USD / kWh that's $1400 / year. When the hardware costs multiple tens of thousands you might be looking at a yearly electricity bill of ~3% of that. And that's before accounting for the datacenter itself. The difference is that much of the hardware crypto was running on was dirt cheap in comparison. Add to that a lot of crypto was being run on discount gear in cobbled together setups. Some people were even stealing electricity which often involved installing large clusters of relatively cheap devices in places without proper cooling.
- jpc0 3mo ago
  Capex != Opex Your are measuring two very different things. And you are only considering electricity bills for the GPU. Higher TDP also means more heat which means more cooling. It's not really easy to make an off hand comment about costs. A 30-40% decreaae in electricity usage and heat output for a fleet of GPUs definitely makes a massive difference to your opex bottom line.
  - dapperdrake 3mo ago
    And the change in failure rate and technical advances are a de-facto change in actual depreciation vs. accounting depreciation.
  - fc417fc802 3mo ago
    The datacenter will already have been built with an explicit cooling budget. Moving 1 watt of heat requires substantially less than 1 watt to do the work. So you've got a constant multiplier against something like a 9% annual failure rate and an order of magnitude difference in cost.
- dapperdrake 3mo ago
  Electricity gets converted to heat which has to be removed through the cooling system. Performance per Watt is the key metric for data centers full of GPUs.
  - fc417fc802 3mo ago
    That just shows up as a constant factor, accounted for at build out. Consider if each watt dissipated required an additional watt. That's clearly an absurd overestimate yet it would "only" double power costs making the example $2800 / year. Underestimate the B200 at $30k USD and you've got 10.5 years of operation before you achieve parity as a fairly unrealistic worst case estimate. Now consider that at a 9% failure rate the average lifespan is only ~11 years. So electricity (both operation and cooling) is pretty much guaranteed to be less than half your total operating cost no matter how you slice it.

Not true. A B200 "only" draws 1 kW. At 16¢ USD / kWh that's $1400 / year. When the hardware costs multiple tens of thousands you might be looking at a yearly electricity bill of ~3% of that. And that's before accounting for the datacenter itself. The difference is that much of the hardware crypto was running on was dirt cheap in comparison. Add to that a lot of crypto was being run on discount gear in cobbled together setups. Some people were even stealing electricity which often involved installing large clusters of relatively cheap devices in places without proper cooling.

jpc0 3mo ago
Capex != Opex Your are measuring two very different things. And you are only considering electricity bills for the GPU. Higher TDP also means more heat which means more cooling. It's not really easy to make an off hand comment about costs. A 30-40% decreaae in electricity usage and heat output for a fleet of GPUs definitely makes a massive difference to your opex bottom line.
- dapperdrake 3mo ago
  And the change in failure rate and technical advances are a de-facto change in actual depreciation vs. accounting depreciation.
- fc417fc802 3mo ago
  The datacenter will already have been built with an explicit cooling budget. Moving 1 watt of heat requires substantially less than 1 watt to do the work. So you've got a constant multiplier against something like a 9% annual failure rate and an order of magnitude difference in cost.
dapperdrake 3mo ago
Electricity gets converted to heat which has to be removed through the cooling system. Performance per Watt is the key metric for data centers full of GPUs.
- fc417fc802 3mo ago
  That just shows up as a constant factor, accounted for at build out. Consider if each watt dissipated required an additional watt. That's clearly an absurd overestimate yet it would "only" double power costs making the example $2800 / year. Underestimate the B200 at $30k USD and you've got 10.5 years of operation before you achieve parity as a fairly unrealistic worst case estimate. Now consider that at a 9% failure rate the average lifespan is only ~11 years. So electricity (both operation and cooling) is pretty much guaranteed to be less than half your total operating cost no matter how you slice it.