3 ms·

> There is absolutely no way anyone is going to be making any money offering $2 H100s unless they stole them and they get free space/power... At the highest po

> There is absolutely no way anyone is going to be making any money offering $2 H100s unless they stole them and they get free space/power... At the highest power settings, H100s consume 400 W. Add another 200 W for CPU/RAM. Assume you have an incredibly inefficient cooling system, so you also need 600 W of cooling. Google tells me US energy prices average around 17 cents/kWh - even if you don't locate your data centre somewhere with cheap electricity. 17 cents/kWh * 1200 watts * 1 hour is only 20.4 cents/hour.

sandworm101 2y ago
For the fuller math one has to include the cost of infrastructure financing, which is tied to interest rates. Given how young most of these H100 shops are, I assume that they pay more to service their debts than for power.
- Wytwwww 2y ago
  > I assume that they pay more to service their debts than for power. Well yes, because for GPU datacentres fixed/capital costs make up a much higher fraction than power and other expenses than for CPUs. To such an extent that power usage barely even matters. A $20k that uses 1 kW ( which is way more than it would in reality ) 24x7 would cost $1.3k to run per year at 0.15$ per kWh, that's almost insignificant compared to depreciation. The premise is that nobody could make any money by renting H100s for 2$ even if they got them for free unless they only had free power. That makes no sense whatsoever when you can get 2x AMD EPYC™ 9454P servers at 2x408 W (for full system) for around $0.70 in a German data center.
ckastner 2y ago
That's just the power. If one expects a H100 to run for three years at full load, 24 x 365 x 3 = 26280. Assuming a price of $25K per H100, that means about $1/h to amortize costs. Hence the unless they stole them, I guess. Factor in space, networking, cooling, security, etc., and $2 really do seem undoable.
- Negitivefrags 2y ago
  None of that matters if you already bought the H100 and have no use for it. You might as well recoup as much money as you can on it.
  - dwattttt 2y ago
    If you already have the H100s, renting access to them at a loss isn't better. Throwing them in the trash will lose you less money.
    - ericpauley 2y ago
      GGP already showed the marginal power cost is well below $2.
      - cheschire 2y ago
        There is so much more to lifecycle sustainment cost than that. Rackspace. Networking. Physical safety. Physical security. Sales staff. Support staff. Legal. Finance. HR. Support staff for those folks. That’s just off the top of my head. Sitting down for a couple days at the very least, like a business should, would likely reveal significant depths that $2 won’t cover.
        H8crilA 2y ago
        So you terminate all of the above right now, or continue selling at a loss (which still extends the runway) and wait for better times? Also, do you know that similar situations occasionally occur in pretty much any market out there? The market doesn't care how much you're losing, it will set a price and it's up to you to take it, or leave it.
        ericpauley 2y ago
        These are all costs of any server hosting business. Other commenters have already shown that $2/hr for a racked 1U server at 400W is perfectly sustainable.
latchkey 2y ago
You are not looking at the full economics of the situation. There are very few data centers left that can do 45kW+ rack density, which translates to 32 H100/MI300x GPUs in a rack. Most datacenters, you're looking at 1 or 2 boxes of 8 GPU, a rack. As a result, it isn't just the price of power, it is whatever the data center wants to charge you. Then you factor in cooling on top of that...

sandworm101 2y ago

For the fuller math one has to include the cost of infrastructure financing, which is tied to interest rates. Given how young most of these H100 shops are, I assume that they pay more to service their debts than for power.

Wytwwww 2y ago
> I assume that they pay more to service their debts than for power. Well yes, because for GPU datacentres fixed/capital costs make up a much higher fraction than power and other expenses than for CPUs. To such an extent that power usage barely even matters. A $20k that uses 1 kW ( which is way more than it would in reality ) 24x7 would cost $1.3k to run per year at 0.15$ per kWh, that's almost insignificant compared to depreciation. The premise is that nobody could make any money by renting H100s for 2$ even if they got them for free unless they only had free power. That makes no sense whatsoever when you can get 2x AMD EPYC™ 9454P servers at 2x408 W (for full system) for around $0.70 in a German data center.

ckastner 2y ago

That's just the power. If one expects a H100 to run for three years at full load, 24 x 365 x 3 = 26280. Assuming a price of $25K per H100, that means about $1/h to amortize costs. Hence the unless they stole them, I guess. Factor in space, networking, cooling, security, etc., and $2 really do seem undoable.

Negitivefrags 2y ago
None of that matters if you already bought the H100 and have no use for it. You might as well recoup as much money as you can on it.
- dwattttt 2y ago
  If you already have the H100s, renting access to them at a loss isn't better. Throwing them in the trash will lose you less money.
  - ericpauley 2y ago
    GGP already showed the marginal power cost is well below $2.
    - cheschire 2y ago
      There is so much more to lifecycle sustainment cost than that. Rackspace. Networking. Physical safety. Physical security. Sales staff. Support staff. Legal. Finance. HR. Support staff for those folks. That’s just off the top of my head. Sitting down for a couple days at the very least, like a business should, would likely reveal significant depths that $2 won’t cover.
      - H8crilA 2y ago
        So you terminate all of the above right now, or continue selling at a loss (which still extends the runway) and wait for better times? Also, do you know that similar situations occasionally occur in pretty much any market out there? The market doesn't care how much you're losing, it will set a price and it's up to you to take it, or leave it.
      - ericpauley 2y ago
        These are all costs of any server hosting business. Other commenters have already shown that $2/hr for a racked 1U server at 400W is perfectly sustainable.

latchkey 2y ago

You are not looking at the full economics of the situation. There are very few data centers left that can do 45kW+ rack density, which translates to 32 H100/MI300x GPUs in a rack. Most datacenters, you're looking at 1 or 2 boxes of 8 GPU, a rack. As a result, it isn't just the price of power, it is whatever the data center wants to charge you. Then you factor in cooling on top of that...