3 ms·

Except that LLMs doesn't benefit from economies of scale. And they don't have that much brand uniqueness, to retain customers, except some hearsay and "vibes".

by Yizahi 1y ago

Except that LLMs doesn't benefit from economies of scale. And they don't have that much brand uniqueness, to retain customers, except some hearsay and "vibes". So if a lot of new free tier customers join it is net negative, because each of their queries has the same load as paid users. And company can't degrade LLM too much, because there is no uniqueness and free customers will just flee to the competitor. I'm thinking that this ClosedAI strategy is not primarily focused on acquiring new independent users, but more focused at making itself deeply entrenched everywhere. So when the "payday" comes and the immense debt will be due, Sam will just ask ask government to bail them out because they would depend on them a lot, and it will. Maybe not directly bail, but provide new investments with favorable terms, etc.

ACCount36 1y ago
What? LLMs do benefit from economies of scale. There are a lot of things like MoE sharding or speculative decoding that only begin to make sense to set up and use when you're dealing with a large inference workload targeting a specific model. That's on top of all the usual datacenter economies of scale. The whole thing with "OpenAI is bleeding money, they'll run out any day now" is pure copium. LLM inference is already profitable for every major provider. They just keep pouring money into infrastructure and R&D - because they expect to be able to build more and more capable systems, and sell more and more inference in the future.
- Yizahi 1y ago
  Single LLM company can't stop investing into better systems and marketing of them, because there is no moat and customers will flee to the ones who do invest. It's free after all. So it is a closed loop which can't be broken, companies can but won't switch to "just inference". And with investing, all of the LLM companies are losing money a lot (on the LLMs specifically).

by Yizahi 1y ago

ACCount36 1y ago

What? LLMs do benefit from economies of scale. There are a lot of things like MoE sharding or speculative decoding that only begin to make sense to set up and use when you're dealing with a large inference workload targeting a specific model. That's on top of all the usual datacenter economies of scale. The whole thing with "OpenAI is bleeding money, they'll run out any day now" is pure copium. LLM inference is already profitable for every major provider. They just keep pouring money into infrastructure and R&D - because they expect to be able to build more and more capable systems, and sell more and more inference in the future.

Yizahi 1y ago
Single LLM company can't stop investing into better systems and marketing of them, because there is no moat and customers will flee to the ones who do invest. It's free after all. So it is a closed loop which can't be broken, companies can but won't switch to "just inference". And with investing, all of the LLM companies are losing money a lot (on the LLMs specifically).