3 ms·
You don’t think GPUs depreciate massively in value? I think they are written off to zero in less than 5 years. And if you look at the ARR of the companies “buy
by PaulRobinson 2mo ago
You don’t think GPUs depreciate massively in value? I think they are written off to zero in less than 5 years.
And if you look at the ARR of the companies “buying” them, I think we can see there’s some significant leverage going on.
- fooker 2mo agoH100, almost a five year old GPU, costs more to buy used now than it was to buy brand new at release. Of course, everything has a lifespan. Consider a simple arithmetic problem, and this mania will start making sense. An H100 costs approximately 30k. You can run a decent latest open model on it at 1000 tokens per second batched. Cost on open router is 4$ per 1m tokens. That's about 120k revenue per year if there's demand. So far, there's unlimited demand. You, as one person, can likely not make the logistics of this work. But this really works with the economies of scale. Now, because of that everyone wants to buy GPUs and we don't have enough. The math works much better with a newer GPU that produces more tokens per second and consumes less energy to do so, even if it costs double. So why would anyone buy an old one? Because demand is orders of magnitude more than supply.