3 ms·
The math seems off to me here. In particular: >> So here is the deal: If you have 16,000 GPUs, and you have a blended average of on-demand (50 percent), one y
by Majromax 2y ago
The math seems off to me here. In particular:
>> So here is the deal: If you have 16,000 GPUs, and you have a blended average of on-demand (50 percent), one year (30 percent), and three year (20 percent) instances, then that works out to $5.27 billion in GPU rental income over four years with an average cost of $9.40 per hour for the H100 GPUs.
This makes a very strong assumption that the rental cost of an H100 will not change over the next four years. This is wildly optimistic. Instead, we can infer expected prices by looking at the differential rates for one and three-year commitments:
>> We estimated the one year reserved instance cost $57.63 per hour, or $7.20 per GPU-hour, and we know that the published price for the three year reserved instance is $43.16, or $5.40 per GPU-hour.
On the margin, the cloud provider should be indifferent between an on-demand rental, a one-year reservation, and a three-year reservation. That implies that three consecutive one-year reservations should provide about the same income as the three-year reservation.^[1]
Someone who places a three-year commitment will spend $16.20 per hour in one year, over three years. The one-year commitment is $7.20 per hour in one year, over one year. Subtract the two, and we get the residual of $9.00, and divide by the two years remaining in the contract to get $4.50.
With this rough calculation, the two-year, one-year forward price of the H100 is about $4.50/hr. If we further assume that the price changes per year with a constant ratio (0.72), we can break that up the per-hour, one-year reservation prices as $7.20 (today), $5.22 (one year from now), and $3.78 (two years from now).
Going further into speculation and applying this ratio to rental revenue on the whole, that "$5.27b over four years" instead becomes $3.47b. Still a reasonable multiple of the purchased infrastructure cost, but it's less outlandish and emphasizes the profit potential of moving first in this sector (getting those long-term commitments while it's still a seller's market).
[1 — I'm ignoring the option-value in the one-year commitment, which allows the customer to seek a better deal after twelve months. This option-value is minimal of the GPU cloud is expected to be at-capacity forever, such that the provider can replace customers easily.]
- fnbr 2y agoThese prices are insane- you can do much, much better than those prices if you negotiate. If you're paying $7.20 per GPU-hour for an H100, you should find another supplier.
- andersa 2y agoYeah, it's absurd. Current market pricing is more like $4.50 for hourly on-demand and around $2 for long term reservations.
- fnbr 2y agoAgreed, that’s what I’m seeing too.
- hi-v-rocknroll 2y agoThe rule-of-thumb over a few decades is that enterprise initial quote pricing is usually 200% of the lowest negotiated price. That final price should be between 9% (PC commodity margins) to 50%+ profit over costs of sales. Also, I really would like to know what the real (unit) costs of goods of H100's would be, but that's proprietary info no one outside of Nvidia execs and accountants would ever see.
- thundergolfer 2y agoThe H100 prices seem wrong based on today's prices. I'm looking at our data on H100 prices across ten cloud providers and the H100 on-demand prices range between $3-6 hr, the average being $3.83 USD. The p5.48xlarge with 8 H100s is indeed $12.29 per GPU-hour as the article states, but their claim that the blended hourly rental price for H100s could be $9.40 USD over the next 4 years appears ludicrous.
- dist-epoch 2y agoThere are already providers today which claim to give you H100 access at $2-4 per hour, with little commitment. So those prices seem wildly optimistic.