3 ms·

> If they stopped today to focus on optimization of their current models to minimize operating cost and monetizing their user base you think they don't have a

> If they stopped today to focus on optimization of their current models to minimize operating cost and monetizing their user base you think they don't have a successful business model? Actually, I'd be very curious to know this. Because we already have a few relatively capable models that I can run on my MBP with 128 GB of RAM (and a few less capable models I can run much faster on my 5090). In order to break even they would have to minimize the operating costs (by throttling, maiming models etc.) and/or increase prices. This would be the reality check. But the cynic in me feels they prefer to avoid this reality check and use the tried and tested Uber model of permanent money influx with the "profitability is just around the corner" justification but at an even bigger scale.

ghc 1y ago
> In order to break even they would have to minimize the operating costs (by throttling, maiming models etc.) and/or increase prices. This would be the reality check. Is that true? Are they operating inference at a loss or are they incurring losses entirely on R&D? I guess we'll probably never know, but I wouldn't take as a given that inference is operating at a loss. I found this: https://semianalysis.com/2023/02/09/the-inference-cost-of-search-disruption/ https://semianalysis.com/2023/02/09/the-inference-cost-of-se... which estimates that it costs $250M/year to operate ChatGPT. If even remotely true $10B in revenue on $250M of COGS would be a great business.
- dvfjsdhgfv 1y ago
  As you say, we will never know, but this article[0] claims: > The cost of the compute to train models alone ($3 billion) obliterates the entirety of its subscription revenue, and the compute from running models ($2 billion) takes the rest, and then some. It doesn’t just cost more to run OpenAI than it makes — it costs the company a billion dollars more than the entirety of its revenue to run the software it sells before any other costs. [0] https://www.lesswrong.com/posts/CCQsQnCMWhJcCFY9x/openai-lost-usd5-billion-in-2024-and-its-losses-are https://www.lesswrong.com/posts/CCQsQnCMWhJcCFY9x/openai-los...
  - ghc 1y ago
    Obviously you don't need to train new models to operate existing ones. I think I trust the semianalysis estimate ($250M) more than this estimate ($2B), but who knows? I do see my revenue estimate was for this year, though. However, $4B revenue on $250M COGS...is still staggeringly good. No wonder amazon, google, and Microsoft are tripping over themselves to offer these models for a fee.
    - hamburga 1y ago
      But assuming no new models are trained, this competitive effect drives down the profit margin on the current SOTA models to zero.
      - ghc 1y ago
        Even if the profit margin is driven to zero, that does not mean competitors will cease to offer the models. It just means the models will be bundled with other services. Case in point: Subversion & Git drove VCS margin to zero (remember BitKeeper?), but Bitbucket and Github wound up becoming good businesses. I think Claude Code might be the start of how companies evolve here.
    - singron 1y ago
      You need to train new models to advance the knowledge cutoff. You don't necessarily need to R&D new architectures, and maybe you can infuse a model with new knowledge without completely training from scratch, but if you do nothing the model will become obsolete. Also the semianalysis estimate is from Feb 2023, which is before the release of gpt4, and it assumes 13 million DAU. ChatGPT has 800 million WAU, so that's somewhere between 115 million and 800 million DAU. E.g. if we prorate the cogs estimate for 200 DAU, then that's 15x higher or $3.75B.
      - ghc 1y ago
        > You need to train new models to advance the knowledge cutoff That's a great point, but I think it's less important now with MCP and RAG. If VC money dried up and the bubble burst, we'd still have broadly useful models that wouldn't be obsolete for years. Releasing a new model every year might be a lot cheaper if a company converts GPU opex to capex and accepts a long training time. > Also the semianalysis estimate is from Feb 2023, Oh! I missed the date. You're right, that's a lot more expensive. On the other hand, inference has likely gotten a lot cheaper (in terms of GPU TOPS) too. Still, I think there's a profitable business model there if VC funding dries up and most of the model companies collapse.

ghc 1y ago

> In order to break even they would have to minimize the operating costs (by throttling, maiming models etc.) and/or increase prices. This would be the reality check. Is that true? Are they operating inference at a loss or are they incurring losses entirely on R&D? I guess we'll probably never know, but I wouldn't take as a given that inference is operating at a loss. I found this: https://semianalysis.com/2023/02/09/the-inference-cost-of-search-disruption/ https://semianalysis.com/2023/02/09/the-inference-cost-of-se... which estimates that it costs $250M/year to operate ChatGPT. If even remotely true $10B in revenue on $250M of COGS would be a great business.

dvfjsdhgfv 1y ago
As you say, we will never know, but this article[0] claims: > The cost of the compute to train models alone ($3 billion) obliterates the entirety of its subscription revenue, and the compute from running models ($2 billion) takes the rest, and then some. It doesn’t just cost more to run OpenAI than it makes — it costs the company a billion dollars more than the entirety of its revenue to run the software it sells before any other costs. [0] https://www.lesswrong.com/posts/CCQsQnCMWhJcCFY9x/openai-lost-usd5-billion-in-2024-and-its-losses-are https://www.lesswrong.com/posts/CCQsQnCMWhJcCFY9x/openai-los...
- ghc 1y ago
  Obviously you don't need to train new models to operate existing ones. I think I trust the semianalysis estimate ($250M) more than this estimate ($2B), but who knows? I do see my revenue estimate was for this year, though. However, $4B revenue on $250M COGS...is still staggeringly good. No wonder amazon, google, and Microsoft are tripping over themselves to offer these models for a fee.
  - hamburga 1y ago
    But assuming no new models are trained, this competitive effect drives down the profit margin on the current SOTA models to zero.
    - ghc 1y ago
      Even if the profit margin is driven to zero, that does not mean competitors will cease to offer the models. It just means the models will be bundled with other services. Case in point: Subversion & Git drove VCS margin to zero (remember BitKeeper?), but Bitbucket and Github wound up becoming good businesses. I think Claude Code might be the start of how companies evolve here.
  - singron 1y ago
    You need to train new models to advance the knowledge cutoff. You don't necessarily need to R&D new architectures, and maybe you can infuse a model with new knowledge without completely training from scratch, but if you do nothing the model will become obsolete. Also the semianalysis estimate is from Feb 2023, which is before the release of gpt4, and it assumes 13 million DAU. ChatGPT has 800 million WAU, so that's somewhere between 115 million and 800 million DAU. E.g. if we prorate the cogs estimate for 200 DAU, then that's 15x higher or $3.75B.
    - ghc 1y ago
      > You need to train new models to advance the knowledge cutoff That's a great point, but I think it's less important now with MCP and RAG. If VC money dried up and the bubble burst, we'd still have broadly useful models that wouldn't be obsolete for years. Releasing a new model every year might be a lot cheaper if a company converts GPU opex to capex and accepts a long training time. > Also the semianalysis estimate is from Feb 2023, Oh! I missed the date. You're right, that's a lot more expensive. On the other hand, inference has likely gotten a lot cheaper (in terms of GPU TOPS) too. Still, I think there's a profitable business model there if VC funding dries up and most of the model companies collapse.