4 ms·
> LLM inference is currently profitable. It depends a lot on which LLMs you're talking about, and what kind of usage. See e.g. the recent post about how "Anthr
by antonvs 1y ago
> LLM inference is currently profitable.
It depends a lot on which LLMs you're talking about, and what kind of usage. See e.g. the recent post about how "Anthropic is bleeding out": https://news.ycombinator.com/item?id=44534291 https://news.ycombinator.com/item?id=44534291
Ignore the hype in the headline, the point is that there's good evidence that inference in many circumstances isn't profitable.
- ctoth 1y ago> In simpler terms, CCusage is a relatively-accurate barometer of how much you are costing Anthropic at any given time, with the understanding that its costs may (we truly have no idea) be lower than the API prices they charge, though I add that based on how Anthropic is expected to lose $3 billion billion this year (that’s after revenue!) there’s a chance that it’s actually losing money on every API call. So he's using their API prices as a proxy for token costs, doesn't actually know the actual inference prices, and ... that's your "good evidence?" This big sentence with all these "We don't knows?"
- antonvs 1y agoWell, that and the $3 billion expected loss after revenue. Does this idea upset you for some reason? Other people have analyzed this and come to similar conclusions, I just picked that one because it's the most recent example I've seen. Feel free to look to a source that explains how LLM Internet is mostly profitable at this point, taking training costs into account. But I suspect you might have a hard time finding evidence of that.
- evertedsphere 1y agowhy'd that get flagged
- antonvs 1y agoFanboys