5 ms·

I think I generally agree with this article, but I have some nitpicks: "OpenAI (...) has yet to create a truly meaningful product" Almost everybody I know use

I think I generally agree with this article, but I have some nitpicks: "OpenAI (...) has yet to create a truly meaningful product" Almost everybody I know uses ChatGPT at least semi-regularly for all kinds of things. I'm not sure if it's possible to look at this objectively and claim that ChatGPT isn't "a truly meaningful product." "OpenAI loses money every single time that somebody uses their product" I'm not sure if this is true. My understanding is that they (on average, including their free users) make money on usage, but lose money on training. But I could be wrong. I also think OpenAI isn't yet serious about monetizing its products. You can just go to ChatGPT and use it for free. They have a ton of data about their users that their users give them freely. The free ChatGPT tier is a prime candidate for advertising.

boxed 2y ago
> make money on usage, but lose money on training. But I could be wrong. You could be 100% right, and it still could be an unsustainable business. Depending on how much training and R&D costs.
- szundi 2y ago
  The more you endure losing money the more of your opponents go bankrupt - or outsmart you
  - consteval 2y ago
    This typically works but the incumbent players - Meta, Google, Microsoft (ish) - are very rich, and are experts. They're no IBM.
scarface_74 2y ago
“Advertising” is not a sure means of profitability. Yahoo was one of the most trafficked website for years and still not profitable.
mossTechnician 2y ago
> My understanding is that they (on average, including their free users) make money on usage, but lose money on training. Doesn't OpenAI need to continue training just to remain relevant? Not even just because there is a race between multiple AI companies, but because new information is regularly published. And if that's the case, it's just part of the cost, right?
- Lerc 2y ago
  There are a few successful game companies that have never turned a profit. Each game brings in more than it cost to make but the development of the following game dwarfs that. I have never been quite sure if it is good practice. It does make you vulnerable to a flop. OpenAI seems to be in a similar position.
  - mossTechnician 2y ago
    As I understand it, OpenAI loses billions of dollars every year, and it receives preferential treatment for cloud credits. Does any gaming company make similar total losses year after year?
    - Lerc 2y ago
      >As I understand it, OpenAI loses billions of dollars every year, and it receives preferential treatment for cloud credits. The absolute values or the deals performed are irrelevant to the principle. The principle is that revenues and expenditure go up in a way that never results in a profit, but if you broke each generation of development into a separate company it would be a series of profitable investments requiring increasingly larger investment for the next company. If serving ChatGPT stays profitable enough to cover the training of the model that they are currently serving then they are financially secure. It will only become a problem if they cannot recover the cost of training each model by serving it. We don't yet know if that will happen. There is insufficient data in this area, Without knowing the capital expenditure on hardware, the costs of running the hardware, and the breakdown between how much of that hardware is running inference and how much is training, we just don't have a reliable way to calculate potential profitability. It's been a long time since GPT4, and all releases since then have been improvements on that generation. Some of those improvements have significantly reduced the cost of inference. There is evidence that they are spending on the next two generations already. We don't yet know how capable the next generation model will be, in the absence of that data point, almost all of the speculation is relatively meaningless. >but I don't see any indicator that OpenAI will not go the route of enshittifiaction if it gets the opportunity There is a very significant indicator. We live in a world where it is commonplace to offer free services paid for by advertising. This is the model where the user is not the customer, it is the product. That is the most common path to enshitification, because pleasing your customer takes priority. OpenAI has not yet taken that path (and they have had the opportunity). Both the API and ChatGPT plus work on the old fashioned model of give us some money and we'll give you a service. That is a model that disincentivises screwing over your customers. It may not stay this way, but they certainly have had opportunities for enshitification that they have chosen not to embrace.
minimaxir 2y ago
> Almost everybody I know uses ChatGPT at least semi-regularly for all kinds of things. Ed's perspective is from a nonengineering side, not the typical Hacker News user or LLM API consumer. > My understanding is that they (on average, including their free users) make money on usage, but lose money on training. The billion-dollar YoY losses can only be possible with negative marginal profits. Training is expensive but not that expensive. > I also think OpenAI isn't yet serious about monetizing its products. You can just go to ChatGPT and use it for free That's just modern software business: have the user try it for free (loss-leading) with an incentive to upsell to a paid product, notably ChatGPT+. And if OpenAI didn't do it, others will as is currently the case now. But in a post ZIRP world that doesn't work as well.
- jessriedel 2y ago
  >> Almost everybody I know uses ChatGPT at least semi-regularly for all kinds of things. > Ed's perspective is from a nonengineering side, not the typical Hacker News user or LLM API consumer. ChatGPT has 200M weekly active users.
  - minimaxir 2y ago
    I'm not saying I agree with him that ChatGPT has no practical use. https://x.com/minimaxir/status/1841549631085543850 https://x.com/minimaxir/status/1841549631085543850
    - jessriedel 2y ago
      I’m responding to the suggestion that being on the non-engineering side explains his opinion.
- jacinabox 2y ago
  One would think that the difficulty of making a company profitable when it's training larger and larger LLMs, combined with the diminishing returns and model collapse phenomenon, would make it so that companies don't wish to stop training larger models. I assume that they continue training larger models because whichever company stops training larger models would fall behind in the race to win new rounds of funding, but if that is the case what is the ultimate valuation that these companies are trying to achieve being valued in the biliions already? Diminishing returns means that the user gets less marginal benefit with each larger model, and the model collapse phenomenon means that models trained on new training data might be less good than older models. Have straightforward mitigations been put in place such as filtering out from the training data forums where users like to share AI generated content?
stouset 2y ago
> Almost everybody I know uses ChatGPT at least semi-regularly for all kinds of things. Count me in the camp that doesn't and wishes fewer people did. I just asked ChatGPT what it knows about me, and apparently I am known for my work in the field of AI. I've apparently contributed to multiple AI projects. I have a globally-unique name and have done absolutely nothing of the sort. I don't dispute that there are good uses for it right now, but people taking this stuff at face value for anything important is terrifying.
- airstrike 2y ago
  "You're holding it wrong." You're likely to get more value out of it if you try to do something useful with it rather than asking it to answer a question you know ex ante it cannot know the answer for.
  - stouset 2y ago
    I expect it to provide information to me on topics I don’t already know. If it can’t do so accurately on topics I do know, what on earth would make me believe it’s any better on everything else? I had absolutely zero reason to believe it would mischaracterize me in advance of asking that question. In fact I presumed that would be an easy gimme: simply Googling me is enough to provide a pretty accurate assessment even before following any of the links.
    - airstrike 2y ago
      > I expect it to provide information to me on topics I don’t already know. Presumably you already know about yourself, yet that was the example you chose to pick. It comes across as you basically leading the witness just to be able to say say "gotcha! I knew you'd suck". It's not a search engine, although you can ask some models/chatbots to search the web. Again, you're holding it wrong. Instead of trying to confirm your bias that LLMs are bad, try to find discomforming evidence. “The human understanding when it has once adopted an opinion, draws all things else to support and agree with it. And though there be a greater number and weight of instances to be found on the other side, yet these it either neglects and despises, or else by some distinction sets aside and rejects.” — Francis Bacon
mvdtnz 2y ago
> Almost everybody I know uses ChatGPT at least semi-regularly for all kinds of things. I simply do not believe you. Most people I speak to attempt to use it periodically, find that for any promising use case it fails completely and then give up for another few weeks.
- InsideOutSanta 2y ago
  "I simply do not believe you." Well, I can't exactly prove it to you. The thing I would offer as evidence, though, is that most of my friends have white-collar jobs. They spend most days sitting in offices writing stuff. Let me give you an example. One of my friends works in a legal department at a bank. She often gets to review things like ads or promotions the bank runs. These often contain things that are illegal, because the marketing team is not made up of people who have a legal background, and many things that seem like reasonable ideas are illegal. So a lot of her work consists of explaining to people why they can't do the thing they want to do. She's not particularly good at doing that in an empathetic way, so she's started using ChatGPT to outline these emails for her. "The marketing team at my bank wants to run a competition where people who open a new account get a chance of winning a prize. This is illegal, since it violates sweepstakes laws (see below, where I've pasted the applicable sections). They either have to provide a way to enter the contest freely, without opening an account, or they have to abandon the idea. Can you write an email that explains this in an empathetic, but clear way?" Obviously, she then has to read the email ChatGPT writes and edit it, but it still saves her time and creates a better end result. "for any promising use case it fails completely" I simply do not believe you :-)
  - mvdtnz 2y ago
    > "The marketing team at my bank wants to run a competition where people who open a new account get a chance of winning a prize. This is illegal, since it violates sweepstakes laws (see below, where I've pasted the applicable sections). They either have to provide a way to enter the contest freely, without opening an account, or they have to abandon the idea. Can you write an email that explains this in an empathetic, but clear way?" Just this paragraph but without the instructions to ChatGPT is sufficient for this case. It's literally more work to get a useful output from ChatGPT than just to do the work yourself. And that's not even accounting for the careful review you need to give every morsel of output from it.

boxed 2y ago

> make money on usage, but lose money on training. But I could be wrong. You could be 100% right, and it still could be an unsustainable business. Depending on how much training and R&D costs.

szundi 2y ago
The more you endure losing money the more of your opponents go bankrupt - or outsmart you
- consteval 2y ago
  This typically works but the incumbent players - Meta, Google, Microsoft (ish) - are very rich, and are experts. They're no IBM.

scarface_74 2y ago

“Advertising” is not a sure means of profitability. Yahoo was one of the most trafficked website for years and still not profitable.

mossTechnician 2y ago

> My understanding is that they (on average, including their free users) make money on usage, but lose money on training. Doesn't OpenAI need to continue training just to remain relevant? Not even just because there is a race between multiple AI companies, but because new information is regularly published. And if that's the case, it's just part of the cost, right?

Lerc 2y ago
There are a few successful game companies that have never turned a profit. Each game brings in more than it cost to make but the development of the following game dwarfs that. I have never been quite sure if it is good practice. It does make you vulnerable to a flop. OpenAI seems to be in a similar position.
- mossTechnician 2y ago
  As I understand it, OpenAI loses billions of dollars every year, and it receives preferential treatment for cloud credits. Does any gaming company make similar total losses year after year?
  - Lerc 2y ago
    >As I understand it, OpenAI loses billions of dollars every year, and it receives preferential treatment for cloud credits. The absolute values or the deals performed are irrelevant to the principle. The principle is that revenues and expenditure go up in a way that never results in a profit, but if you broke each generation of development into a separate company it would be a series of profitable investments requiring increasingly larger investment for the next company. If serving ChatGPT stays profitable enough to cover the training of the model that they are currently serving then they are financially secure. It will only become a problem if they cannot recover the cost of training each model by serving it. We don't yet know if that will happen. There is insufficient data in this area, Without knowing the capital expenditure on hardware, the costs of running the hardware, and the breakdown between how much of that hardware is running inference and how much is training, we just don't have a reliable way to calculate potential profitability. It's been a long time since GPT4, and all releases since then have been improvements on that generation. Some of those improvements have significantly reduced the cost of inference. There is evidence that they are spending on the next two generations already. We don't yet know how capable the next generation model will be, in the absence of that data point, almost all of the speculation is relatively meaningless. >but I don't see any indicator that OpenAI will not go the route of enshittifiaction if it gets the opportunity There is a very significant indicator. We live in a world where it is commonplace to offer free services paid for by advertising. This is the model where the user is not the customer, it is the product. That is the most common path to enshitification, because pleasing your customer takes priority. OpenAI has not yet taken that path (and they have had the opportunity). Both the API and ChatGPT plus work on the old fashioned model of give us some money and we'll give you a service. That is a model that disincentivises screwing over your customers. It may not stay this way, but they certainly have had opportunities for enshitification that they have chosen not to embrace.

minimaxir 2y ago

> Almost everybody I know uses ChatGPT at least semi-regularly for all kinds of things. Ed's perspective is from a nonengineering side, not the typical Hacker News user or LLM API consumer. > My understanding is that they (on average, including their free users) make money on usage, but lose money on training. The billion-dollar YoY losses can only be possible with negative marginal profits. Training is expensive but not that expensive. > I also think OpenAI isn't yet serious about monetizing its products. You can just go to ChatGPT and use it for free That's just modern software business: have the user try it for free (loss-leading) with an incentive to upsell to a paid product, notably ChatGPT+. And if OpenAI didn't do it, others will as is currently the case now. But in a post ZIRP world that doesn't work as well.

jessriedel 2y ago
>> Almost everybody I know uses ChatGPT at least semi-regularly for all kinds of things. > Ed's perspective is from a nonengineering side, not the typical Hacker News user or LLM API consumer. ChatGPT has 200M weekly active users.
- minimaxir 2y ago
  I'm not saying I agree with him that ChatGPT has no practical use. https://x.com/minimaxir/status/1841549631085543850 https://x.com/minimaxir/status/1841549631085543850
  - jessriedel 2y ago
    I’m responding to the suggestion that being on the non-engineering side explains his opinion.
jacinabox 2y ago
One would think that the difficulty of making a company profitable when it's training larger and larger LLMs, combined with the diminishing returns and model collapse phenomenon, would make it so that companies don't wish to stop training larger models. I assume that they continue training larger models because whichever company stops training larger models would fall behind in the race to win new rounds of funding, but if that is the case what is the ultimate valuation that these companies are trying to achieve being valued in the biliions already? Diminishing returns means that the user gets less marginal benefit with each larger model, and the model collapse phenomenon means that models trained on new training data might be less good than older models. Have straightforward mitigations been put in place such as filtering out from the training data forums where users like to share AI generated content?

stouset 2y ago

> Almost everybody I know uses ChatGPT at least semi-regularly for all kinds of things. Count me in the camp that doesn't and wishes fewer people did. I just asked ChatGPT what it knows about me, and apparently I am known for my work in the field of AI. I've apparently contributed to multiple AI projects. I have a globally-unique name and have done absolutely nothing of the sort. I don't dispute that there are good uses for it right now, but people taking this stuff at face value for anything important is terrifying.

airstrike 2y ago
"You're holding it wrong." You're likely to get more value out of it if you try to do something useful with it rather than asking it to answer a question you know ex ante it cannot know the answer for.
- stouset 2y ago
  I expect it to provide information to me on topics I don’t already know. If it can’t do so accurately on topics I do know, what on earth would make me believe it’s any better on everything else? I had absolutely zero reason to believe it would mischaracterize me in advance of asking that question. In fact I presumed that would be an easy gimme: simply Googling me is enough to provide a pretty accurate assessment even before following any of the links.
  - airstrike 2y ago
    > I expect it to provide information to me on topics I don’t already know. Presumably you already know about yourself, yet that was the example you chose to pick. It comes across as you basically leading the witness just to be able to say say "gotcha! I knew you'd suck". It's not a search engine, although you can ask some models/chatbots to search the web. Again, you're holding it wrong. Instead of trying to confirm your bias that LLMs are bad, try to find discomforming evidence. “The human understanding when it has once adopted an opinion, draws all things else to support and agree with it. And though there be a greater number and weight of instances to be found on the other side, yet these it either neglects and despises, or else by some distinction sets aside and rejects.” — Francis Bacon

mvdtnz 2y ago

> Almost everybody I know uses ChatGPT at least semi-regularly for all kinds of things. I simply do not believe you. Most people I speak to attempt to use it periodically, find that for any promising use case it fails completely and then give up for another few weeks.

InsideOutSanta 2y ago
"I simply do not believe you." Well, I can't exactly prove it to you. The thing I would offer as evidence, though, is that most of my friends have white-collar jobs. They spend most days sitting in offices writing stuff. Let me give you an example. One of my friends works in a legal department at a bank. She often gets to review things like ads or promotions the bank runs. These often contain things that are illegal, because the marketing team is not made up of people who have a legal background, and many things that seem like reasonable ideas are illegal. So a lot of her work consists of explaining to people why they can't do the thing they want to do. She's not particularly good at doing that in an empathetic way, so she's started using ChatGPT to outline these emails for her. "The marketing team at my bank wants to run a competition where people who open a new account get a chance of winning a prize. This is illegal, since it violates sweepstakes laws (see below, where I've pasted the applicable sections). They either have to provide a way to enter the contest freely, without opening an account, or they have to abandon the idea. Can you write an email that explains this in an empathetic, but clear way?" Obviously, she then has to read the email ChatGPT writes and edit it, but it still saves her time and creates a better end result. "for any promising use case it fails completely" I simply do not believe you :-)
- mvdtnz 2y ago
  > "The marketing team at my bank wants to run a competition where people who open a new account get a chance of winning a prize. This is illegal, since it violates sweepstakes laws (see below, where I've pasted the applicable sections). They either have to provide a way to enter the contest freely, without opening an account, or they have to abandon the idea. Can you write an email that explains this in an empathetic, but clear way?" Just this paragraph but without the instructions to ChatGPT is sufficient for this case. It's literally more work to get a useful output from ChatGPT than just to do the work yourself. And that's not even accounting for the careful review you need to give every morsel of output from it.