3 ms·

This is addressed in the article. Giving arguments for llms being profitable as APIs.

n4r9 1y ago
One of those arguments is: > there's not that much motive to gain API market share with unsustainably cheap prices. Any gains would be temporary, since there's no long-term lock-in, and better models are released weekly The goal may be not so much locking customers in, but outlasting other LLM providers whilst maintaining a good brand image. Once everyone starts seeing you as "the" LLM provider, costs can start going up. That's what Uber and Lyft have been trying to do (though obviously without success). Also, the prices may become more sustainable if LLM providers find ways to inject ad revenue into their products.
- unilynx 1y ago
  > Also, the prices may become more sustainable if LLM providers find ways to inject ad revenue into their products. I'm sure they've already found ways to do that, injecting relevant ads is just a form of RAG. But they won't risk it yet as long as they're still grabbing market share just like Google didn't run them at the start - and kept them unobtrusive until their search won.
- pr337h4m 1y ago
  Uber and Lyft rely on network effects, which do not exist in any meaningful sense for LLM API providers.
  - nitwit005 1y ago
    Brand is huge in every market. It's hard to get people to visit your website at all. People know about OpenAI, and look it up.
    - TZubiri 1y ago
      No network effect + already profitable. Not at all like Uber, let it go
      - nitwit005 1y ago
        Incoherent response.
  - n4r9 1y ago
    Yeah, that's definitely a factor in the attempt to "undercut and outlast". I guess I have two defenses: firstly, network effects might not be crucial, it might be enough for there to be a small cost to changing provider; secondly, I imagine the providers are finding ways to use network effects to bolster adoption - e.g. "Find me a party date when all my friends are free, book the catering and message them with invites".
username223 1y ago
It's addressed poorly. > First, there's not that much motive to gain API market share with unsustainably cheap prices. Any gains would be temporary, since there's no long-term lock-in, What? If someone builds something on top of your API, they're tying themselves to it, and you can slowly raise prices while keeping each increase well below the switching cost. > Second, some of those models have been released with open weights and API access is also available from third-party providers who would have no motive to subsidize inference. See above. Just like any other Cloud service, you tie clients to your API. > Third, Deepseek released actual numbers on their inference efficiency in February. Those numbers suggest that their normal R1 API pricing has about 80% margins when considering the GPU costs, though not any other serving costs. 80% margin on GPU cost? What about after paying for power, facilities, admin, support, marketing, etc.? Are GPUs really more than half the cost of this business? (EDIT: This is 80% margin on top of GPU rental, i.e. total compute cost. My bad.) Guessing about costs based on prices makes no sense at this point. OpenAI's $20/mo and $200/mo tiers have nothing to do with the cost of those services -- they're just testing price points.
- petesergeant 1y ago
  > What? If someone builds something on top of your API, they're tying themselves to it, and you can slowly raise prices while keeping each increase well below the switching cost. Have you used any of these APIs? There's very little lock-in for inference. This isn't like setting up all your automation on S3, if you use the right library it's changing a config file.
- jsnell 1y ago
  > What? If someone builds something on top of your API, they're tying themselves to it, and you can slowly raise prices while keeping each increase well below the switching cost. That's not really how the LLM API market works. The interfaces themselves are pretty trivial and have no real lock-in value, and there's plenty of adapters around anyway. (Often first-party, e.g. both Anthropic and Google provide OpenAI-compatible APIs). There might initially have been theories that you could not easily move to a different model, creating lock-in, but in practice LLMs are so flexible and forgiving about the inputs that a different model can be just dropped in an work without any model-specific changes. > 80% margin on GPU cost? What about after paying for power, facilities The market price of renting that compute on the market. That's fully loaded, so would include a) pro-rated recouping the capital cost of the GPUs, b) the power, cooling, datacenter buildings, etc, c) the hosting provider's margin. > admin, support, marketing, etc.? Are GPUs really more than half the cost of this business? Pretty likely! In OpenAI's leaked 2024 financial plan the compute costs were like 75% of their projected costs.

n4r9 1y ago

One of those arguments is: > there's not that much motive to gain API market share with unsustainably cheap prices. Any gains would be temporary, since there's no long-term lock-in, and better models are released weekly The goal may be not so much locking customers in, but outlasting other LLM providers whilst maintaining a good brand image. Once everyone starts seeing you as "the" LLM provider, costs can start going up. That's what Uber and Lyft have been trying to do (though obviously without success). Also, the prices may become more sustainable if LLM providers find ways to inject ad revenue into their products.

unilynx 1y ago
> Also, the prices may become more sustainable if LLM providers find ways to inject ad revenue into their products. I'm sure they've already found ways to do that, injecting relevant ads is just a form of RAG. But they won't risk it yet as long as they're still grabbing market share just like Google didn't run them at the start - and kept them unobtrusive until their search won.
pr337h4m 1y ago
Uber and Lyft rely on network effects, which do not exist in any meaningful sense for LLM API providers.
- nitwit005 1y ago
  Brand is huge in every market. It's hard to get people to visit your website at all. People know about OpenAI, and look it up.
  - TZubiri 1y ago
    No network effect + already profitable. Not at all like Uber, let it go
    - nitwit005 1y ago
      Incoherent response.
- n4r9 1y ago
  Yeah, that's definitely a factor in the attempt to "undercut and outlast". I guess I have two defenses: firstly, network effects might not be crucial, it might be enough for there to be a small cost to changing provider; secondly, I imagine the providers are finding ways to use network effects to bolster adoption - e.g. "Find me a party date when all my friends are free, book the catering and message them with invites".

username223 1y ago

It's addressed poorly. > First, there's not that much motive to gain API market share with unsustainably cheap prices. Any gains would be temporary, since there's no long-term lock-in, What? If someone builds something on top of your API, they're tying themselves to it, and you can slowly raise prices while keeping each increase well below the switching cost. > Second, some of those models have been released with open weights and API access is also available from third-party providers who would have no motive to subsidize inference. See above. Just like any other Cloud service, you tie clients to your API. > Third, Deepseek released actual numbers on their inference efficiency in February. Those numbers suggest that their normal R1 API pricing has about 80% margins when considering the GPU costs, though not any other serving costs. 80% margin on GPU cost? What about after paying for power, facilities, admin, support, marketing, etc.? Are GPUs really more than half the cost of this business? (EDIT: This is 80% margin on top of GPU rental, i.e. total compute cost. My bad.) Guessing about costs based on prices makes no sense at this point. OpenAI's $20/mo and $200/mo tiers have nothing to do with the cost of those services -- they're just testing price points.

petesergeant 1y ago
> What? If someone builds something on top of your API, they're tying themselves to it, and you can slowly raise prices while keeping each increase well below the switching cost. Have you used any of these APIs? There's very little lock-in for inference. This isn't like setting up all your automation on S3, if you use the right library it's changing a config file.
jsnell 1y ago
> What? If someone builds something on top of your API, they're tying themselves to it, and you can slowly raise prices while keeping each increase well below the switching cost. That's not really how the LLM API market works. The interfaces themselves are pretty trivial and have no real lock-in value, and there's plenty of adapters around anyway. (Often first-party, e.g. both Anthropic and Google provide OpenAI-compatible APIs). There might initially have been theories that you could not easily move to a different model, creating lock-in, but in practice LLMs are so flexible and forgiving about the inputs that a different model can be just dropped in an work without any model-specific changes. > 80% margin on GPU cost? What about after paying for power, facilities The market price of renting that compute on the market. That's fully loaded, so would include a) pro-rated recouping the capital cost of the GPUs, b) the power, cooling, datacenter buildings, etc, c) the hosting provider's margin. > admin, support, marketing, etc.? Are GPUs really more than half the cost of this business? Pretty likely! In OpenAI's leaked 2024 financial plan the compute costs were like 75% of their projected costs.