3 ms·
Some basic math supports it. A GB300 NVL72 is about $6.5 million. Lets say that you need $6 million worth of cooling and another $6 million worth of electricity
by selectodude 5mo ago
Some basic math supports it. A GB300 NVL72 is about $6.5 million. Lets say that you need $6 million worth of cooling and another $6 million worth of electricity. At current rates, that's 720 billion tokens worth of Claude Opus 4.7. At 100,000 tokens per second, it pays for itself in about 3 months.
Obviously this is an extremely rough calculation. I can even be off by a factor of 10 and it's still a pretty good return.
- overfeed 5mo agoUnless you're serving Chinese open-weight models - you have to consoder training costs. If you're off my 10x, then the amortization period is 30 months - far longer than the useful lifetimes of SoTA models. Frontier model development is a Red Queens race: you have to run as fast as you can, just to maintain your position.
- selectodude 5mo agoThe discussion was if Anthropic makes money on inference. They do. They lose billions on training.
- joshuastuden 5mo agoNo, because Anthropic can't serve their models unless they train them. Training is akin to the cost of building the software/product. Inference is selling the product.
- BadBadJellyBean 5mo agoIt's quite easy to sell something for a profit if you ignore the costs. Ultimate free money hack. I will start selling canned beans for the price of the beans plus a few cents. I will just ignore the cost of the cans, labor, power, machines, maintenance, distribution, storage and facility space. If I do that the few cents extra are pure profit.