4 ms·

I don’t know why everyone’s confident that the investments won’t pay off. Every such post in HN is such a way. 800 million weekly active users from ChatGPT impl

I don’t know why everyone’s confident that the investments won’t pay off. Every such post in HN is such a way. 800 million weekly active users from ChatGPT implies that people actually like LLM’s and likely the growth will still keep increasing. Every signal points to this - so investment in data centers make sense?

catmanjan 1y ago
Imagine how many millions would drive a Ferrari if they gave them away for free
- simianwords 1y ago
  It’s not true that people are only using it because it’s free. It’s actually quite interesting to see these contradictory positions play out: 1. LLMs are useless and everyone is making a stupid bet on it. The users of llms are fooled into using it and the companies are fooled into betting on it 2. Llms are getting so cheap that the investments into data centers won’t pay off because apparently they will get good enough to run on your phone 3. Llms are bad and they are bad for environment, bad for the brain, bad because they displace workers and bad because they make rich people richer 4. AI is only kept up because there’s a conspiracy to keep it propped up by Nvidia, oracle, OpenAI (something something circular economy) 5. AI is so powerful that it should not be built or humanity would go extinct
  - catmanjan 1y ago
    It is true that none of the LLM providers are profitable though, so there is some number above free that they need to charge and I am not convinced that number is compelling
    - simianwords 1y ago
      None of LLM providers being profitable is exactly the situation I would expect. Them being profitable is so absurd on the contrary! Why wouldn't they put the money back into R&D and marketing?
      - catmanjan 1y ago
        I'm not well versed with the accountant terminology, whatever the word is to describe the operating cost, I am not convinced consumers will ever pay enough to cover those costs
        simianwords 1y ago
        Do you think if LLM's become 10 times more efficient it might covert he costs? What efficiency increase would you think is enough?
        snovv_crash 1y ago
        It's a competitive environment, no way the data centers manage to capture that 10x efficiency improvement. There would be an expectation of 10x reduced prices, because someone else is offering that.
        catmanjan 1y ago
        The problem I see as someone who has implemented a bunch of AI solutions in a range of markets, the quality isn't good enough yet to even think about efficiency - even if the current AI is 100x more efficient it still wouldn't be worth paying for because it doesn't deliver reliable and trustable results...
  - snovv_crash 1y ago
    A) Huge straw man, since it isn't the same people making those points. None of those need the other to be true to cause issues, they are independent concerns. B) You're missing a few things like: 1. The hardware overhang of edge compute (especially phones) may make the centralized compute investments irrelevant as more efficient LLMs (or whatever replaces them) are released. 2. Hardware depreciates quickly. Are these massive data centers really going to earn their money back before a more efficient architecture makes them obsolete? Look at all the NPUs on phones which are useless with most current LLMs due to insufficient RAM. Maybe analogue compute takes off, or giant FPGAs, which can do on a single board what is done with a rack at the moment. We are nowhere near a stable model architecture, or stable optimal compute architecture. Follow the trajectory of bitcoin and etherium mining here to see what we can expect. 3. How does one company earn back their R&D when the moment it is released, competition puts out comparable models within 6 months, possibly by using the very service that was provided to generate training data.
- re-thc 1y ago
  > Imagine how many millions would drive a Ferrari if they gave them away for free 0. Ferrari is a luxury sports brand. What's the point of it if it flooded the streets?
  - high_na_euv 1y ago
    Good looking, powerful, reliable car?
    - fragmede 1y ago
      > Good looking, powerful, reliable car? How to say you don't own a Ferrari without saying you don't own a Ferrari.
techpression 1y ago
They like them when they’re free and/or cheap. But will that be sustainable? The answer to that question is far less certain. Maybe Ads will save them, maybe royalties, maybe price hikes. But it’s far from certain at least.
- simianwords 1y ago
  I really don’t get this. People actually pay and use and enjoy llms. My company has paid for Gemini and ChatGPT subscriptions and people actually use them. Why do you automatically assume that people won’t pay for it?
  - fabian2k 1y ago
    Because they probably would have to pay a lot more to make this profitable at the levels current valuations and investments indicate. And the barrier for private persons to pay is much larger than for companies. I don't think anyone has a really solid handle on the economics here yet, as the field is changing very quickly. But there is a big difference here compared to most software companies. The product does cost significant money per additional customer and usage. There is a real product here. And you can likely earn money with it. But the question is "how much money?", and whether these huge data center investments will actually pay off.
    - simianwords 1y ago
      > Because they probably would have to pay a lot more to make this profitable at the levels current valuations and investments indicate. I keep hearing this but this is very unlikely to be true. The cost of LLMs have gone down by more than 30 times in the past 1 year. How much more should it go down until you consider it economically feasible?
      - fabian2k 1y ago
        Why are they building so many data centers then? That is all cost that has to be earned back. And using them as agents creates much higher costs per interactions than just chatting. We also don't know if the current prices are in any way economical, or how they are related to actual development and interference costs.
matusp 1y ago
The question is if it is economically feasible. People can enjoy generating funny AI images to share with their friends, but it might not be economically feasible to invest $100B to give them this toy. There is a question of how much value using GenAI generates for the economy.
- simianwords 1y ago
  The cost of LLMs have gone down over 30 times in the last 1-2 years. At what point would you think it is economically feasible? I think this a question to ask so that we can tackle the fundamental economics of LLMs.
  - ehnto 1y ago
    We would also need to know how many people are willing to pay for it, on a consumer level. Additionally business will need to see real ROI for using it. At the moment everyone is trying their best to implement it, but it remains to be clear if it actually increases a company's profitability. Time will tell, and I think there are a lot of things obfuscating the reality right now that the market will eventually make clear. Additionally the economics of training new models may not work out, another detail that's currently obfuscated by venture capital footing the bill.
    - deleted 1y ago
      [deleted]
  - matusp 1y ago
    It becomes feasible when people are willing to pay more than it costs to run it. But I think this will be a pretty uphill battle, as many use cases are hard to monetize (eg proofreading) and for many use cases you will feel pressure from smaller models (you don't need the most expensive SOTA model to generate an email). There is probably just a very limited amount of use cases that are in the goldilocks zone with their difficulty so that people will be willing to pay for them, AND AI is able to solve them. I think programming might be one of them.
wowohwow 1y ago
History doesn't repeat itself, but it rhymes. Callbacks to the infrastructure laid out during dotcom bubble, i.e Cisco, and all of the networking infrastructure. Likewise, internal memos at Oracle indicate they're losing money on their hardware. Anecdotal and hand wavy; but there are plenty of signals out there that it won't pay off. I'm not arguing one way or the other, but there are plenty of arguments for it to not succeed in a way that's required to justify the mind boggling growth we've been experiencing.
- re-thc 1y ago
  > internal memos at Oracle indicate they're losing money on their hardware That doesn't mean much does it? Oracle is a huge company. Not just a cloud either. Companies often offer discounts or promotions; so? There could be plenty of managed services, managed databases, CRM and many more that make up for it. Whilst I'm not sure if Oracle's stock price is right - the memo was more like a way to pressure the stock down for whatever reason.
- simianwords 1y ago
  The mind boggling growth you should also focus on is consumer demand of LLMs. Can you account for it and then re-analyse?
  - overfeed 1y ago
    Surely, Oracle could make up for its unit losses with volume, just like SNL's "Change Bank"
    - simianwords 1y ago
      why do you assume unit losses? they can just price it a bit higher
    - wolfie69 1y ago
      Lol CityWide Change Bank. I was nervous because I was running out of change, then I saw a CityWide Change Bank up ahead. She went to CityWide Change Bank to get exact change for the toll, "to save time".
grues-dinner 1y ago
Building the datacentres may be massively "productive" in GDP terms (servers! GPU! Power plants! Electrical inspection!), but when they're completed, that activity will cease. Datacentres don't employ many people directly and mostly consume only electricity. So if the action of datacentre building shows up as essentially the only GDP growth, but what later happens in the datacentres fails to take its place or exceed it, there will be a dip. Whether LLMs grinding away can prop up all GDP growth from now on remains to be seen. People use them when they're free, but people also collected AOL discs for tree decorations because they were free.
- simianwords 1y ago
  You don’t seem to have got my point. You say the investment won’t pay off but why not? This is quite a big assumption to make considering there’s huge evidence that people like LLMs.
  - csoups14 1y ago
    You don't think it's a bigger assumption that plowing hundreds of billions of dollars into a novel and often misunderstood technology is going to net out positive? We're not talking "people like feeds of cat photos" money and risk here, we're talking about a bubble with the potential to tank the entire economy. On top of that, it's also a higher disruptive technology that should it ramp as quickly as it might will lead to massive amounts of societal strife at a time where we're already stretched a little thin to say the least. "What do you mean Credit Default Swaps don't work? They've been functioning perfectly well these past few years and so far the impacts have only been positive. Tons of people are getting mortgages on houses they weren't in the past, it's the American dream!"
  - grues-dinner 1y ago
    I'm not saying they won't (the "if" is an important word in the sentence), it's just that datacentre growth is not the same as AI growth. It's related, but it is, by definition, temporary as datacentres are soon completed. There's obviously evidence people use LLMs. That's not necessarily the same as people paying a noticeable fraction of all their money to use them in perpetuity. And even if "normal" people do start taking out $50 subscriptions as a matter of course, commoditisation could push that price down as could "dumping" of cheap models from overseas. A breakthrough in being able to run "good enough" models very cheaply, or even locally, would also make expensive cloud AI subscriptions a hard sell. And expensive subscriptions are the only way this pans out. It hasn't yet been shown that AI is a gas that will fill all the available capacity, and keep it filled. If bread were 10 times cheaper, would you buy 10x as much? That has more or less happened to food availability in the West over the last 200 years and OpenBread and BunVidia don't dominate the economy. None of that is sure to happen, and maybe the AI hype train is right and huge expensive LLMs specifically drive a gigantic productivity boom¹ and are worth, say, 0.2*GDP forever. But if it isn't, and it turns out $5 a month gets you all people actually need, it's going to be untidy. ¹: in which case, why is GDP not growing from the AI we already have?
schnitzelstoat 1y ago
Yeah, I don't think ChatGPT is going to take all our jobs, but it might replace Google for a lot of searches. That alone will be a monumental shakeup for the industry.
csomar 1y ago
We have hit a plateau for many months for the performance of LLMs. Anthropic recently released 4.5, and while it improved on some contexts, it failed to make a commit message for me a few times on a workflow I had. 3.5 to 4 had close to zero failure rates on this workflow and it was surprising to see 4.5 fail. It seems that gains in certain benchmarks will affect quality elsewhere. LLMs are very useful, I can’t see myself walking back to the old way of doing things. But the amounts invested expect major breakthrough that we are not anywhere near. It’s a gamble and that’s what innovation is; but you gamble on a small portion of your wealth. Not your house and certainly you do not gamble a huge country like the US on a single thing.
throwaway290 1y ago
"People actually like LLM's" People also like pizza. How many million weekly active consumers of pizza? how about rice? Really what people like here is cheap stuff and having a job that pays money to buy it. chatgpt so far loses boatloads of money. Soon they jack up prices, add adds, and people realize that it was all trained on them & threatens their job. So really right now chatgpt is sweating hard to make itself too big to fail. https://news.ycombinator.com/item?id=45511368 https://news.ycombinator.com/item?id=45511368
nylonstrung 1y ago
Forget profitability, the major Western LLM providers still have profoundly negative gross margins I think it will only be economically sound as a business if you're Google and can start serving ads OR when we switch over from GPUs to wildly more efficient TPUs/ASICs All the data center CapEx is going into compute that will be obsolete once that happens
nylonstrung 1y ago
Forget profitability, the major Western LLM providers still have profoundly negative gross margins. What would that 800M userbase look like if free tier users had to pay the bill for inference and training costs? I think it will only be economically sound as a business if you're Google and can start serving ads OR when we switch over from GPUs to wildly more efficient TPUs/ASICs All the data center CapEx is going into compute that will be obsolete once that happens
lm28469 1y ago
> 800 million weekly active users from ChatGPT implies that people actually like LLM’s 800m total users, 25m paying customers... Most people use free accounts and would likely never pay any substantial amount of money for them https://www.theverge.com/openai/640894/chatgpt-has-hit-20-million-paid-subscribers?ref=wheresyoured.at https://www.theverge.com/openai/640894/chatgpt-has-hit-20-mi...

catmanjan 1y ago

Imagine how many millions would drive a Ferrari if they gave them away for free

simianwords 1y ago
It’s not true that people are only using it because it’s free. It’s actually quite interesting to see these contradictory positions play out: 1. LLMs are useless and everyone is making a stupid bet on it. The users of llms are fooled into using it and the companies are fooled into betting on it 2. Llms are getting so cheap that the investments into data centers won’t pay off because apparently they will get good enough to run on your phone 3. Llms are bad and they are bad for environment, bad for the brain, bad because they displace workers and bad because they make rich people richer 4. AI is only kept up because there’s a conspiracy to keep it propped up by Nvidia, oracle, OpenAI (something something circular economy) 5. AI is so powerful that it should not be built or humanity would go extinct
- catmanjan 1y ago
  It is true that none of the LLM providers are profitable though, so there is some number above free that they need to charge and I am not convinced that number is compelling
  - simianwords 1y ago
    None of LLM providers being profitable is exactly the situation I would expect. Them being profitable is so absurd on the contrary! Why wouldn't they put the money back into R&D and marketing?
    - catmanjan 1y ago
      I'm not well versed with the accountant terminology, whatever the word is to describe the operating cost, I am not convinced consumers will ever pay enough to cover those costs
      - simianwords 1y ago
        Do you think if LLM's become 10 times more efficient it might covert he costs? What efficiency increase would you think is enough?
        snovv_crash 1y ago
        It's a competitive environment, no way the data centers manage to capture that 10x efficiency improvement. There would be an expectation of 10x reduced prices, because someone else is offering that.
        catmanjan 1y ago
        The problem I see as someone who has implemented a bunch of AI solutions in a range of markets, the quality isn't good enough yet to even think about efficiency - even if the current AI is 100x more efficient it still wouldn't be worth paying for because it doesn't deliver reliable and trustable results...
- snovv_crash 1y ago
  A) Huge straw man, since it isn't the same people making those points. None of those need the other to be true to cause issues, they are independent concerns. B) You're missing a few things like: 1. The hardware overhang of edge compute (especially phones) may make the centralized compute investments irrelevant as more efficient LLMs (or whatever replaces them) are released. 2. Hardware depreciates quickly. Are these massive data centers really going to earn their money back before a more efficient architecture makes them obsolete? Look at all the NPUs on phones which are useless with most current LLMs due to insufficient RAM. Maybe analogue compute takes off, or giant FPGAs, which can do on a single board what is done with a rack at the moment. We are nowhere near a stable model architecture, or stable optimal compute architecture. Follow the trajectory of bitcoin and etherium mining here to see what we can expect. 3. How does one company earn back their R&D when the moment it is released, competition puts out comparable models within 6 months, possibly by using the very service that was provided to generate training data.
re-thc 1y ago
> Imagine how many millions would drive a Ferrari if they gave them away for free 0. Ferrari is a luxury sports brand. What's the point of it if it flooded the streets?
- high_na_euv 1y ago
  Good looking, powerful, reliable car?
  - fragmede 1y ago
    > Good looking, powerful, reliable car? How to say you don't own a Ferrari without saying you don't own a Ferrari.

techpression 1y ago

They like them when they’re free and/or cheap. But will that be sustainable? The answer to that question is far less certain. Maybe Ads will save them, maybe royalties, maybe price hikes. But it’s far from certain at least.

simianwords 1y ago
I really don’t get this. People actually pay and use and enjoy llms. My company has paid for Gemini and ChatGPT subscriptions and people actually use them. Why do you automatically assume that people won’t pay for it?
- fabian2k 1y ago
  Because they probably would have to pay a lot more to make this profitable at the levels current valuations and investments indicate. And the barrier for private persons to pay is much larger than for companies. I don't think anyone has a really solid handle on the economics here yet, as the field is changing very quickly. But there is a big difference here compared to most software companies. The product does cost significant money per additional customer and usage. There is a real product here. And you can likely earn money with it. But the question is "how much money?", and whether these huge data center investments will actually pay off.
  - simianwords 1y ago
    > Because they probably would have to pay a lot more to make this profitable at the levels current valuations and investments indicate. I keep hearing this but this is very unlikely to be true. The cost of LLMs have gone down by more than 30 times in the past 1 year. How much more should it go down until you consider it economically feasible?
    - fabian2k 1y ago
      Why are they building so many data centers then? That is all cost that has to be earned back. And using them as agents creates much higher costs per interactions than just chatting. We also don't know if the current prices are in any way economical, or how they are related to actual development and interference costs.

matusp 1y ago

The question is if it is economically feasible. People can enjoy generating funny AI images to share with their friends, but it might not be economically feasible to invest $100B to give them this toy. There is a question of how much value using GenAI generates for the economy.

simianwords 1y ago
The cost of LLMs have gone down over 30 times in the last 1-2 years. At what point would you think it is economically feasible? I think this a question to ask so that we can tackle the fundamental economics of LLMs.
- ehnto 1y ago
  We would also need to know how many people are willing to pay for it, on a consumer level. Additionally business will need to see real ROI for using it. At the moment everyone is trying their best to implement it, but it remains to be clear if it actually increases a company's profitability. Time will tell, and I think there are a lot of things obfuscating the reality right now that the market will eventually make clear. Additionally the economics of training new models may not work out, another detail that's currently obfuscated by venture capital footing the bill.
  - deleted 1y ago
    [deleted]
- matusp 1y ago
  It becomes feasible when people are willing to pay more than it costs to run it. But I think this will be a pretty uphill battle, as many use cases are hard to monetize (eg proofreading) and for many use cases you will feel pressure from smaller models (you don't need the most expensive SOTA model to generate an email). There is probably just a very limited amount of use cases that are in the goldilocks zone with their difficulty so that people will be willing to pay for them, AND AI is able to solve them. I think programming might be one of them.

wowohwow 1y ago

History doesn't repeat itself, but it rhymes. Callbacks to the infrastructure laid out during dotcom bubble, i.e Cisco, and all of the networking infrastructure. Likewise, internal memos at Oracle indicate they're losing money on their hardware. Anecdotal and hand wavy; but there are plenty of signals out there that it won't pay off. I'm not arguing one way or the other, but there are plenty of arguments for it to not succeed in a way that's required to justify the mind boggling growth we've been experiencing.

re-thc 1y ago
> internal memos at Oracle indicate they're losing money on their hardware That doesn't mean much does it? Oracle is a huge company. Not just a cloud either. Companies often offer discounts or promotions; so? There could be plenty of managed services, managed databases, CRM and many more that make up for it. Whilst I'm not sure if Oracle's stock price is right - the memo was more like a way to pressure the stock down for whatever reason.
simianwords 1y ago
The mind boggling growth you should also focus on is consumer demand of LLMs. Can you account for it and then re-analyse?
- overfeed 1y ago
  Surely, Oracle could make up for its unit losses with volume, just like SNL's "Change Bank"
  - simianwords 1y ago
    why do you assume unit losses? they can just price it a bit higher
  - wolfie69 1y ago
    Lol CityWide Change Bank. I was nervous because I was running out of change, then I saw a CityWide Change Bank up ahead. She went to CityWide Change Bank to get exact change for the toll, "to save time".

grues-dinner 1y ago

Building the datacentres may be massively "productive" in GDP terms (servers! GPU! Power plants! Electrical inspection!), but when they're completed, that activity will cease. Datacentres don't employ many people directly and mostly consume only electricity. So if the action of datacentre building shows up as essentially the only GDP growth, but what later happens in the datacentres fails to take its place or exceed it, there will be a dip. Whether LLMs grinding away can prop up all GDP growth from now on remains to be seen. People use them when they're free, but people also collected AOL discs for tree decorations because they were free.

simianwords 1y ago
You don’t seem to have got my point. You say the investment won’t pay off but why not? This is quite a big assumption to make considering there’s huge evidence that people like LLMs.
- csoups14 1y ago
  You don't think it's a bigger assumption that plowing hundreds of billions of dollars into a novel and often misunderstood technology is going to net out positive? We're not talking "people like feeds of cat photos" money and risk here, we're talking about a bubble with the potential to tank the entire economy. On top of that, it's also a higher disruptive technology that should it ramp as quickly as it might will lead to massive amounts of societal strife at a time where we're already stretched a little thin to say the least. "What do you mean Credit Default Swaps don't work? They've been functioning perfectly well these past few years and so far the impacts have only been positive. Tons of people are getting mortgages on houses they weren't in the past, it's the American dream!"
- grues-dinner 1y ago
  I'm not saying they won't (the "if" is an important word in the sentence), it's just that datacentre growth is not the same as AI growth. It's related, but it is, by definition, temporary as datacentres are soon completed. There's obviously evidence people use LLMs. That's not necessarily the same as people paying a noticeable fraction of all their money to use them in perpetuity. And even if "normal" people do start taking out $50 subscriptions as a matter of course, commoditisation could push that price down as could "dumping" of cheap models from overseas. A breakthrough in being able to run "good enough" models very cheaply, or even locally, would also make expensive cloud AI subscriptions a hard sell. And expensive subscriptions are the only way this pans out. It hasn't yet been shown that AI is a gas that will fill all the available capacity, and keep it filled. If bread were 10 times cheaper, would you buy 10x as much? That has more or less happened to food availability in the West over the last 200 years and OpenBread and BunVidia don't dominate the economy. None of that is sure to happen, and maybe the AI hype train is right and huge expensive LLMs specifically drive a gigantic productivity boom¹ and are worth, say, 0.2*GDP forever. But if it isn't, and it turns out $5 a month gets you all people actually need, it's going to be untidy. ¹: in which case, why is GDP not growing from the AI we already have?

schnitzelstoat 1y ago

Yeah, I don't think ChatGPT is going to take all our jobs, but it might replace Google for a lot of searches. That alone will be a monumental shakeup for the industry.

csomar 1y ago

We have hit a plateau for many months for the performance of LLMs. Anthropic recently released 4.5, and while it improved on some contexts, it failed to make a commit message for me a few times on a workflow I had. 3.5 to 4 had close to zero failure rates on this workflow and it was surprising to see 4.5 fail. It seems that gains in certain benchmarks will affect quality elsewhere. LLMs are very useful, I can’t see myself walking back to the old way of doing things. But the amounts invested expect major breakthrough that we are not anywhere near. It’s a gamble and that’s what innovation is; but you gamble on a small portion of your wealth. Not your house and certainly you do not gamble a huge country like the US on a single thing.

throwaway290 1y ago

"People actually like LLM's" People also like pizza. How many million weekly active consumers of pizza? how about rice? Really what people like here is cheap stuff and having a job that pays money to buy it. chatgpt so far loses boatloads of money. Soon they jack up prices, add adds, and people realize that it was all trained on them & threatens their job. So really right now chatgpt is sweating hard to make itself too big to fail. https://news.ycombinator.com/item?id=45511368 https://news.ycombinator.com/item?id=45511368

nylonstrung 1y ago

Forget profitability, the major Western LLM providers still have profoundly negative gross margins I think it will only be economically sound as a business if you're Google and can start serving ads OR when we switch over from GPUs to wildly more efficient TPUs/ASICs All the data center CapEx is going into compute that will be obsolete once that happens

nylonstrung 1y ago

Forget profitability, the major Western LLM providers still have profoundly negative gross margins. What would that 800M userbase look like if free tier users had to pay the bill for inference and training costs? I think it will only be economically sound as a business if you're Google and can start serving ads OR when we switch over from GPUs to wildly more efficient TPUs/ASICs All the data center CapEx is going into compute that will be obsolete once that happens

lm28469 1y ago

> 800 million weekly active users from ChatGPT implies that people actually like LLM’s 800m total users, 25m paying customers... Most people use free accounts and would likely never pay any substantial amount of money for them https://www.theverge.com/openai/640894/chatgpt-has-hit-20-million-paid-subscribers?ref=wheresyoured.at https://www.theverge.com/openai/640894/chatgpt-has-hit-20-mi...