3 ms·
They are taking in more than they are spending hosting them. However, the cost for training the next generation of models is not covered.
by hobom 6mo ago
They are taking in more than they are spending hosting them. However, the cost for training the next generation of models is not covered.
- bandrami 6mo agoNope. They're losing money on straight inference (you may be thinking of the interview where Dario described a hypothetical company that was positive margin). The only way they can make it look like they're making money on inference is by calling the ongoing reinforcement training of the currently-served model a capital rather than operational expense, which is both absurd and will absolutely not work for an IPO.
- dgellow 6mo agoDo you have sources? I would be interested to read them
- bandrami 6mo agoProbably the best roundup is Ed Zitron at https://wheresyoured.at https://wheresyoured.at Half the articles are paywalled but the free ones outline the financial situation of the SOTA providers and he has receipts
- wild_egg 6mo agoInference, in and of itself, can't be completely unprofitable. Unless you're purely talking about Anthropic? But > If you want LLMs to continue to be offered we have to get to a point where the providers are taking in more money than they are spending hosting them Suggests you just mean in general, as a category, every provider is taking a loss. That seems implausible. Every provider on OpenRouter is giving away inference at a loss? For what purpose?
- bandrami 6mo agoFor the same reason that Amazon operated at a loss for two decades and Uber operated at a loss for a decade and a half. The problem is the free money hose isn't running anymore.
- victorbjorklund 5mo agoI really doubt that since prices are even higher than no-name hosts on open router etc charges.