12 ms·

Everyone is trying to compare AI companies with something that happened in the past, but I don't think we can predict much from that. GPUs are not railroads or

Everyone is trying to compare AI companies with something that happened in the past, but I don't think we can predict much from that. GPUs are not railroads or fiber optics. The cost structure of ChatGPT and other LLM based services is entirely different than web, they are very expensive to build but also cost a lot to serve. Companies like Meta, Microsoft, Amazon, Google would all survive if their massive investment does not pay off. On the other hand, OpenAI, Anthropic and others could be soon find themselves in a difficult position and be at the mercy of Nvidia.

yieldcrv 1y ago
Just because they have ongoing costs after purchasing them doesn't mean it's different than something else we've seen? What are you trying to articulate exactly, this is a simple business and can get costs under control eventually, or not
wood_spirit 1y ago
Unlike railroads and fibre, all the best compute in 2025 will be lacklustre in 2027. It won’t retain much value in the same way as the infrastructure of previous bubbles did?
- Analemma_ 1y ago
  Exactly: when was the last time you used ChatGPT-3.5? Its value deprecated to zero after, what, two-and-a-half years? (And the Nvidia chips used to train it have barely retained any value either) The financials here are so ugly: you have to light truckloads of money on fire forever just to jog in place.
  - mattmanser 1y ago
    But is it a bit like a game of musical chairs? At some point the AI becomes good enough, and if you're not sitting in a chair at the time, you're not going to be the next Google.
    - potatolicious 1y ago
      Not necessarily? That assumes that the first "good enough" model is a defensible moat - i.e., the first ones to get there becomes the sole purveyors of the Good AI. In practice that hasn't borne out. You can download and run open weight models now that are spitting distance to state-of-the-art, and open weight models are at best a few months behind the proprietary stuff. And even within the realm of proprietary models no player can maintain a lead. Any advances are rapidly matched by the other players. More likely at some point the AI becomes "good enough"... and every single player will also get a "good enough" AI shortly thereafter. There doesn't seem like there's a scenario where any player can afford to stop setting cash on fire and start making money.
      - wood_spirit 1y ago
        Perhaps the first thing the owners ask the first true AGI is “how do I dominate the world?” and the AGI outlines how to stop any competitor getting AGI..?
  - falcor84 1y ago
    I would think that it's more like a general codebase - even if after 2.5 years, 95% percent of the lines were rewritten, and even if the whole thing was rewritten in a different language, there is no point in time at which its value diminished, as you arguably couldn't have built the new version without all the knowledge (and institutional knowledge) from the older version.
    - spwa4 1y ago
      I rejoined an previous employer of mine, someone everyone here knows ... and I found that half their networking equipment is still being maintained by code I wrote in 2012-2014. It has not been rewritten. Hell, I rewrote a few parts that badly needed it despite joining another part of the company.
  - cj 1y ago
    > money on fire forever just to jog in place. Why? I don't see why these companies can't just stop training at some point. Unless you're saying the cost of inference is unsustainable? I can envision a future where ChatGPT stops getting new SOTA models, and all future models are built for enterprise or people willing to pay a lot of money for high ROI use cases. We don't need better models for the vast majority of chats taking place today E.g. kids using it for help with homework - are today's models really not good enough?
    - Eisenstein 1y ago
      They aren't. They are obsequious. This is much worse than it seems at first glance, and you can tell it is a big deal because a lot of effort going into training the new models is to mitigate it.
    - MontyCarloHall 1y ago
      >I don't see why these companies can't just stop training at some point. Because training isn't just about making brand new models with better capabilities, it's also about updating old models to stay current with new information. Even the most sophisticated present-day model with a knowledge cutoff date of 2025 would be severely crippled by 2027 and utterly useless by 2030. Unless there is some breakthrough that lets existing models cheaply incrementally update their weights to add new information, I don't see any way around this.
      - fennecbutt 1y ago
        Ain't never hearda rag
        MontyCarloHall 1y ago
        There is no evidence that RAG delivers equivalent performance to retraining on new data. Merely having information in the context window is very different from having it baked into the model weights. Relying solely on RAG to keep model results current would also degrade with time, as more and more information would have to be incorporated into the context window the longer it's been since the knowledge cutoff date.
  - fooker 1y ago
    > And the Nvidia chips used to train it have barely retained any value either Oh, I'd love to get a cheap H100! Where can I find one? You'll find it costs almost as much used as it's new.
    - counterargument 1y ago
      [dead]
  - tim333 1y ago
    OpenAI is now valued at $500bn though. I doubt the investors are too wrecked yet. It may be like looking at the early Google and saying they are spending loads on compute and haven't even figured how to monetize search, the investors are doomed.
    - oblio 1y ago
      Google was founded in 1998 and IPOed in 2004. If OpenAI was feeling confident they'd find ways to set up a company and IPO, 9 years after founding. It's all mostly fictional money at this point.
      - matwood 1y ago
        It's not about confidence. OpenAI would be huge on the public markets, but since they can raise plenty of money in the private market there is no reason to deal with that hassle - yet.
      - aurareturn 1y ago
        If OpenAI is a public company today, I would bet almost anything that it'd be a $1+ trillion company immediately on opening day.
  - CompoundEyes 1y ago
    A really did few days ago gpt-3.5-fast is a great model for certain tasks and cost wise via the API. Lots of solutions being built on the today’s latest are for tomorrow’s legacy model — if it works just pin the version.
- potatolicious 1y ago
  Yep, we are (unfortunately) still running on railroad infrastructure built a century ago. The amortization periods on that spending is ridiculously long. Effectively every single H100 in existence now will be e-waste in 5 years or less. Not exactly railroad infrastructure here, or even dark fiber.
  - fooker 1y ago
    > Effectively every single H100 in existence now will be e-waste in 5 years or less. This remains to be seen. H100 is 3 years old now, and is still the workhorse of all the major AI shops. When there's something that is obviously better for training, these are still going to be used for inference. If what you say is true, you could find a A100 for cheap/free right now. But check out the prices.
    - fxtentacle 1y ago
      Yeah, I can rent an A100 server for roughly the same price as what the electricity would cost me.
      - typpilol 1y ago
        Where?
        Cheer2171 1y ago
        ~$1.25-1.75/hr at Runpod or vast.ai for an A100 Edit: https://getdeploying.com/reference/cloud-gpu/nvidia-a100 https://getdeploying.com/reference/cloud-gpu/nvidia-a100
        diziet 1y ago
        The A100 SXM4 has a TDP of 400 watts, let's say about 800 with cooling etc overhead. Bulk pricing per KWH is about 8-9 cents industrial. We're over an order of magnitude off here. At 20k per card all in price (MSRSP + datacenter costs) for the 80GB version, with a 4 year payoff schedule the card costs 57 cents per hour (20,000/24/365/4) assuming 100% utilization.
- layoric 1y ago
  > Unlike railroads and fibre, all the best compute in 2025 will be lacklustre in 2027. I definitely don't think compute is anything like railroads and fibre, but I'm not so sure compute will continue it's efficiency gains of the past. Power consumption for these chips is climbing fast, lots of gains are from better hardware support for 8bit/4bit precision, I believe yields are getting harder to achieve as things get much smaller. Betting against compute getting better/cheaper/faster is probably a bad idea, but fundamental improvements I think will be a lot slower over the next decade as shrinking gets a lot harder.
  - spiderice 1y ago
    Unfortunately changing 2027 to 2030 doesn't make the math much better
    - JumpCrisscross 1y ago
      > changing 2027 to 2030 doesn't make the math much better Could you show me? Early turbines didn't last that long. Even modern ones are only rated for a few decades.
      - singularity2001 1y ago
        there is a difference between a few decades and half a decade though? Or his time in general accelerated so much that it's basically very similar
        JumpCrisscross 1y ago
        “Within Parsons' lifetime, the generating capacity of a [steam turbine] unit was scaled up by about 10,000 times” [1]. For comparison, Moore’s law (at 2 years per doubling) scales 4 orders of magnitude in about 27 years. That’s roughly the lifetime of a modern steam turbine [2]. In actuality, Parsons lived 77 years [3], implying a 13% growth rate, so doubling every 6 versus 2 years. But within the same order of magnitude. [1] https://en.m.wikipedia.org/wiki/Steam_turbine https://en.m.wikipedia.org/wiki/Steam_turbine [2] https://alliedpg.com/latest-articles/life-extension-strategies-aging-steam-turbines/ https://alliedpg.com/latest-articles/life-extension-strategi... 30 years [3] https://en.m.wikipedia.org/wiki/Charles_Algernon_Parsons https://en.m.wikipedia.org/wiki/Charles_Algernon_Parsons
- christina97 1y ago
  The A100 came out 5.5 years ago and is still the staple for many AI/ML workloads. Even AI hardware just doesn’t depreciate that quickly.
  - dzhiurgis 1y ago
    This. There’s even a market for them being built (DRW).
  - oblio 1y ago
    Don't they degrade physically from being run at full blast 24/7 for so many years?
    - the-grump 1y ago
      Not nearly as badly if you keep them cool.
  - Ianjit 1y ago
    Users are waiting for Blackwell. Then Rubin. CRWV depreciates GPUs over 6 years. Rails last a lot longer.
  - littlestymaar 1y ago
    Unless you care about FLOP/Watt, which big players definitely do.
    - christina97 1y ago
      No more than 5-10% of the hourly cost of an A100 will be in actual power. It’s not a big factor.
- Spooky23 1y ago
  Unlike 1875, we have Saudi and other tillion/billionaires willing commit almost any amount to own the future of business.
  - rchaud 1y ago
    Except they behave less like shrewd investors and more like bandwagon jumpers looking to buy influence or get rich quick. Crypto, Twitter, ridesharing, office sharing and now AI. None of these have been the future of business. Business looks a lot like what it has throughout history. Building physical transport infrastructure, trade links, improving agricultural and manufacturing productivity and investing in military advancements. In the latter respect, countries like Turkey and Iran are decades ahead of Saudi in terms of building internal security capacity with drone tech for example.
    - Spooky23 1y ago
      Agreed - I don’t think they are particularly brilliant as a category. Hereditary kleptocracy has limits. But… I don’t think there’s an example in modern history of the this much capital moving around based on whim. The “bet on red” mentality has produced some odd leaders with absolute authority in their domain. One of the most influential figures on the US government claims to believe that he is saving society from the antichrist. Another thinks he’s the protagonist in a sci-fi novel. We have the madness of monarchy with modern weapons and power. Yikes.
- mcswell 1y ago
  "...all the best compute in 2025 will be lacklustre in 2027": How does the compute (I assume you mean on PCs) of 2025 compare with the compute of 2023? Oh wait, the computer I'm typing this on was manufactured in 2020...
  - brianwawok 1y ago
    Neato. How’s that 1999 era laptop? Because 25 year old trains are still running and 25 year old train track is still almost new. It’s not the same and you know it.
    - 1oooqooq 1y ago
      last month HN was talking about a win95 with floppy drivers handling rail in Germany no less
    - mcswell 1y ago
      What's that 26 year difference have to do with it? The part I quoted was talking about 2027 vs. 2025, a two year difference.
JCM9 1y ago
Businesses are different but the fundamentals of business and finance stay consistent. In every bubble that reality is unavoidable, no matter how much people say/wish “but this time is different.”
lossolo 1y ago
The funniest thing about all this is that the biggest difference between LLMs from Anthropic, Google, OpenAI, Alibaba is not model architecture or training objectives, which are broadly similar but it's the dataset. What people don't realize is how much of that data comes from massive undisclosed scrapes + synthetic data + countless hours of expert feedback shaping the models. As methodologies converge, the performance gap between these systems is already narrowing and will continue to diminish over time.
LarsDu88 1y ago
If you build the actual datacenter, less than half the cost is the actual compute. The other half is the actual datacenter infrastructure, power infrastructure, and cooling. So in that sense it's not that much different from Meta and Google which also used server infrastructure that depreciated over time. The difference is that I believe Meta and Google made money hand over fist even in their earliest days.
- Lalo-ATX 1y ago
  Last time i ran the numbers - Data center facilities are ~$10k per kW IT gear is like $20k-$50k per kW Data center gear is good for 15-30 years. IT is like 2-6ish. Would love to see updated numbers. Got any?
conartist6 1y ago
It's not that the investments just won't pay off, it's that the global markets are likely to crash like happened with the subprime mortgage crisis.
- mothballed 1y ago
  The fed could still push the real value of stocks quite a bit by destroying the USD, if they want, by pinning interest rates near 0 and forcing a rush to the exits to buy stock and other asset classes.
  - mcny 1y ago
    The point still stands though. All these other companies can pivot to some thing else if AI fails but what will OpenAI do?
    - mandeepj 1y ago
      > but what will OpenAI do? Will get acquired at “Store Closing” price!!
    - rglover 1y ago
      Sell to Microsoft and be absorbed there (and Anthropic to Amazon).
    - rubyfan 1y ago
      By the time it catches up with them they will have IPO’d and dumped their problem onto the public market. The administration will probably get a golden share and they will get a bail out in an effort to soften the landing for their campaign donors that also have huge positions. All the rich people will be made whole and the US tax payer will pay the price of the bail out. And Microsoft or whoever will absorb the remains of their technology.
- vitaflo 1y ago
  This is much closer to the dotcom boom than the subprime stuff. The dotcom boom/bust affected tech more than anything else. It didn’t involve consumers like the housing crash did.
redwood 1y ago
I'm reminded of the quote "If you owe the bank $100 that's your problem. If you owe the bank $100 million, that's the bank's problem." - J. Paul Getty Nvidia may well be at the mercy of them! Hence the recent circular dealing
bee_rider 1y ago
The past/present company they remind me of the most is semiconductor fabs. Significant generation-to-generation R&D investment, significant hardware and infrastructure investment, quite winner-takes-all on the high end, obsoleted in a couple years at most. The main differences are these models are early in their development curve so the jumps are much bigger, and they are entirely digital so they get “shipped” much faster, and open weights seem to be possible. None of those factors seem to make it a more attractive business to be in.
EasyMark 1y ago
In the end Revenues > Costs or you have an issue. That "startup" money will eventually be gone, and you're back to MIMO Money In vs Money Out and if it's not > , you will go bankrupt.
01100011 1y ago
The one thing smaller companies might have is allocated power budgets from power companies. Part of the mad dash to build datacenters right now is just to claim the power so your competitors can't. Now I do think the established players hold an edge here, but I don't think OpenAI/Anthropic/etc are without some bargaining power(hah).

yieldcrv 1y ago

Just because they have ongoing costs after purchasing them doesn't mean it's different than something else we've seen? What are you trying to articulate exactly, this is a simple business and can get costs under control eventually, or not

wood_spirit 1y ago

Unlike railroads and fibre, all the best compute in 2025 will be lacklustre in 2027. It won’t retain much value in the same way as the infrastructure of previous bubbles did?

Analemma_ 1y ago
Exactly: when was the last time you used ChatGPT-3.5? Its value deprecated to zero after, what, two-and-a-half years? (And the Nvidia chips used to train it have barely retained any value either) The financials here are so ugly: you have to light truckloads of money on fire forever just to jog in place.
- mattmanser 1y ago
  But is it a bit like a game of musical chairs? At some point the AI becomes good enough, and if you're not sitting in a chair at the time, you're not going to be the next Google.
  - potatolicious 1y ago
    Not necessarily? That assumes that the first "good enough" model is a defensible moat - i.e., the first ones to get there becomes the sole purveyors of the Good AI. In practice that hasn't borne out. You can download and run open weight models now that are spitting distance to state-of-the-art, and open weight models are at best a few months behind the proprietary stuff. And even within the realm of proprietary models no player can maintain a lead. Any advances are rapidly matched by the other players. More likely at some point the AI becomes "good enough"... and every single player will also get a "good enough" AI shortly thereafter. There doesn't seem like there's a scenario where any player can afford to stop setting cash on fire and start making money.
    - wood_spirit 1y ago
      Perhaps the first thing the owners ask the first true AGI is “how do I dominate the world?” and the AGI outlines how to stop any competitor getting AGI..?
- falcor84 1y ago
  I would think that it's more like a general codebase - even if after 2.5 years, 95% percent of the lines were rewritten, and even if the whole thing was rewritten in a different language, there is no point in time at which its value diminished, as you arguably couldn't have built the new version without all the knowledge (and institutional knowledge) from the older version.
  - spwa4 1y ago
    I rejoined an previous employer of mine, someone everyone here knows ... and I found that half their networking equipment is still being maintained by code I wrote in 2012-2014. It has not been rewritten. Hell, I rewrote a few parts that badly needed it despite joining another part of the company.
- cj 1y ago
  > money on fire forever just to jog in place. Why? I don't see why these companies can't just stop training at some point. Unless you're saying the cost of inference is unsustainable? I can envision a future where ChatGPT stops getting new SOTA models, and all future models are built for enterprise or people willing to pay a lot of money for high ROI use cases. We don't need better models for the vast majority of chats taking place today E.g. kids using it for help with homework - are today's models really not good enough?
  - Eisenstein 1y ago
    They aren't. They are obsequious. This is much worse than it seems at first glance, and you can tell it is a big deal because a lot of effort going into training the new models is to mitigate it.
  - MontyCarloHall 1y ago
    >I don't see why these companies can't just stop training at some point. Because training isn't just about making brand new models with better capabilities, it's also about updating old models to stay current with new information. Even the most sophisticated present-day model with a knowledge cutoff date of 2025 would be severely crippled by 2027 and utterly useless by 2030. Unless there is some breakthrough that lets existing models cheaply incrementally update their weights to add new information, I don't see any way around this.
    - fennecbutt 1y ago
      Ain't never hearda rag
      - MontyCarloHall 1y ago
        There is no evidence that RAG delivers equivalent performance to retraining on new data. Merely having information in the context window is very different from having it baked into the model weights. Relying solely on RAG to keep model results current would also degrade with time, as more and more information would have to be incorporated into the context window the longer it's been since the knowledge cutoff date.
- fooker 1y ago
  > And the Nvidia chips used to train it have barely retained any value either Oh, I'd love to get a cheap H100! Where can I find one? You'll find it costs almost as much used as it's new.
  - counterargument 1y ago
    [dead]
- tim333 1y ago
  OpenAI is now valued at $500bn though. I doubt the investors are too wrecked yet. It may be like looking at the early Google and saying they are spending loads on compute and haven't even figured how to monetize search, the investors are doomed.
  - oblio 1y ago
    Google was founded in 1998 and IPOed in 2004. If OpenAI was feeling confident they'd find ways to set up a company and IPO, 9 years after founding. It's all mostly fictional money at this point.
    - matwood 1y ago
      It's not about confidence. OpenAI would be huge on the public markets, but since they can raise plenty of money in the private market there is no reason to deal with that hassle - yet.
    - aurareturn 1y ago
      If OpenAI is a public company today, I would bet almost anything that it'd be a $1+ trillion company immediately on opening day.
- CompoundEyes 1y ago
  A really did few days ago gpt-3.5-fast is a great model for certain tasks and cost wise via the API. Lots of solutions being built on the today’s latest are for tomorrow’s legacy model — if it works just pin the version.
potatolicious 1y ago
Yep, we are (unfortunately) still running on railroad infrastructure built a century ago. The amortization periods on that spending is ridiculously long. Effectively every single H100 in existence now will be e-waste in 5 years or less. Not exactly railroad infrastructure here, or even dark fiber.
- fooker 1y ago
  > Effectively every single H100 in existence now will be e-waste in 5 years or less. This remains to be seen. H100 is 3 years old now, and is still the workhorse of all the major AI shops. When there's something that is obviously better for training, these are still going to be used for inference. If what you say is true, you could find a A100 for cheap/free right now. But check out the prices.
  - fxtentacle 1y ago
    Yeah, I can rent an A100 server for roughly the same price as what the electricity would cost me.
    - typpilol 1y ago
      Where?
      - Cheer2171 1y ago
        ~$1.25-1.75/hr at Runpod or vast.ai for an A100 Edit: https://getdeploying.com/reference/cloud-gpu/nvidia-a100 https://getdeploying.com/reference/cloud-gpu/nvidia-a100
        diziet 1y ago
        The A100 SXM4 has a TDP of 400 watts, let's say about 800 with cooling etc overhead. Bulk pricing per KWH is about 8-9 cents industrial. We're over an order of magnitude off here. At 20k per card all in price (MSRSP + datacenter costs) for the 80GB version, with a 4 year payoff schedule the card costs 57 cents per hour (20,000/24/365/4) assuming 100% utilization.
layoric 1y ago
> Unlike railroads and fibre, all the best compute in 2025 will be lacklustre in 2027. I definitely don't think compute is anything like railroads and fibre, but I'm not so sure compute will continue it's efficiency gains of the past. Power consumption for these chips is climbing fast, lots of gains are from better hardware support for 8bit/4bit precision, I believe yields are getting harder to achieve as things get much smaller. Betting against compute getting better/cheaper/faster is probably a bad idea, but fundamental improvements I think will be a lot slower over the next decade as shrinking gets a lot harder.
- spiderice 1y ago
  Unfortunately changing 2027 to 2030 doesn't make the math much better
  - JumpCrisscross 1y ago
    > changing 2027 to 2030 doesn't make the math much better Could you show me? Early turbines didn't last that long. Even modern ones are only rated for a few decades.
    - singularity2001 1y ago
      there is a difference between a few decades and half a decade though? Or his time in general accelerated so much that it's basically very similar
      - JumpCrisscross 1y ago
        “Within Parsons' lifetime, the generating capacity of a [steam turbine] unit was scaled up by about 10,000 times” [1]. For comparison, Moore’s law (at 2 years per doubling) scales 4 orders of magnitude in about 27 years. That’s roughly the lifetime of a modern steam turbine [2]. In actuality, Parsons lived 77 years [3], implying a 13% growth rate, so doubling every 6 versus 2 years. But within the same order of magnitude. [1] https://en.m.wikipedia.org/wiki/Steam_turbine https://en.m.wikipedia.org/wiki/Steam_turbine [2] https://alliedpg.com/latest-articles/life-extension-strategies-aging-steam-turbines/ https://alliedpg.com/latest-articles/life-extension-strategi... 30 years [3] https://en.m.wikipedia.org/wiki/Charles_Algernon_Parsons https://en.m.wikipedia.org/wiki/Charles_Algernon_Parsons
christina97 1y ago
The A100 came out 5.5 years ago and is still the staple for many AI/ML workloads. Even AI hardware just doesn’t depreciate that quickly.
- dzhiurgis 1y ago
  This. There’s even a market for them being built (DRW).
- oblio 1y ago
  Don't they degrade physically from being run at full blast 24/7 for so many years?
  - the-grump 1y ago
    Not nearly as badly if you keep them cool.
- Ianjit 1y ago
  Users are waiting for Blackwell. Then Rubin. CRWV depreciates GPUs over 6 years. Rails last a lot longer.
- littlestymaar 1y ago
  Unless you care about FLOP/Watt, which big players definitely do.
  - christina97 1y ago
    No more than 5-10% of the hourly cost of an A100 will be in actual power. It’s not a big factor.
Spooky23 1y ago
Unlike 1875, we have Saudi and other tillion/billionaires willing commit almost any amount to own the future of business.
- rchaud 1y ago
  Except they behave less like shrewd investors and more like bandwagon jumpers looking to buy influence or get rich quick. Crypto, Twitter, ridesharing, office sharing and now AI. None of these have been the future of business. Business looks a lot like what it has throughout history. Building physical transport infrastructure, trade links, improving agricultural and manufacturing productivity and investing in military advancements. In the latter respect, countries like Turkey and Iran are decades ahead of Saudi in terms of building internal security capacity with drone tech for example.
  - Spooky23 1y ago
    Agreed - I don’t think they are particularly brilliant as a category. Hereditary kleptocracy has limits. But… I don’t think there’s an example in modern history of the this much capital moving around based on whim. The “bet on red” mentality has produced some odd leaders with absolute authority in their domain. One of the most influential figures on the US government claims to believe that he is saving society from the antichrist. Another thinks he’s the protagonist in a sci-fi novel. We have the madness of monarchy with modern weapons and power. Yikes.
mcswell 1y ago
"...all the best compute in 2025 will be lacklustre in 2027": How does the compute (I assume you mean on PCs) of 2025 compare with the compute of 2023? Oh wait, the computer I'm typing this on was manufactured in 2020...
- brianwawok 1y ago
  Neato. How’s that 1999 era laptop? Because 25 year old trains are still running and 25 year old train track is still almost new. It’s not the same and you know it.
  - 1oooqooq 1y ago
    last month HN was talking about a win95 with floppy drivers handling rail in Germany no less
  - mcswell 1y ago
    What's that 26 year difference have to do with it? The part I quoted was talking about 2027 vs. 2025, a two year difference.

JCM9 1y ago

Businesses are different but the fundamentals of business and finance stay consistent. In every bubble that reality is unavoidable, no matter how much people say/wish “but this time is different.”

lossolo 1y ago

The funniest thing about all this is that the biggest difference between LLMs from Anthropic, Google, OpenAI, Alibaba is not model architecture or training objectives, which are broadly similar but it's the dataset. What people don't realize is how much of that data comes from massive undisclosed scrapes + synthetic data + countless hours of expert feedback shaping the models. As methodologies converge, the performance gap between these systems is already narrowing and will continue to diminish over time.

LarsDu88 1y ago

If you build the actual datacenter, less than half the cost is the actual compute. The other half is the actual datacenter infrastructure, power infrastructure, and cooling. So in that sense it's not that much different from Meta and Google which also used server infrastructure that depreciated over time. The difference is that I believe Meta and Google made money hand over fist even in their earliest days.

Lalo-ATX 1y ago
Last time i ran the numbers - Data center facilities are ~$10k per kW IT gear is like $20k-$50k per kW Data center gear is good for 15-30 years. IT is like 2-6ish. Would love to see updated numbers. Got any?

conartist6 1y ago

It's not that the investments just won't pay off, it's that the global markets are likely to crash like happened with the subprime mortgage crisis.

mothballed 1y ago
The fed could still push the real value of stocks quite a bit by destroying the USD, if they want, by pinning interest rates near 0 and forcing a rush to the exits to buy stock and other asset classes.
- mcny 1y ago
  The point still stands though. All these other companies can pivot to some thing else if AI fails but what will OpenAI do?
  - mandeepj 1y ago
    > but what will OpenAI do? Will get acquired at “Store Closing” price!!
  - rglover 1y ago
    Sell to Microsoft and be absorbed there (and Anthropic to Amazon).
  - rubyfan 1y ago
    By the time it catches up with them they will have IPO’d and dumped their problem onto the public market. The administration will probably get a golden share and they will get a bail out in an effort to soften the landing for their campaign donors that also have huge positions. All the rich people will be made whole and the US tax payer will pay the price of the bail out. And Microsoft or whoever will absorb the remains of their technology.
vitaflo 1y ago
This is much closer to the dotcom boom than the subprime stuff. The dotcom boom/bust affected tech more than anything else. It didn’t involve consumers like the housing crash did.

redwood 1y ago

I'm reminded of the quote "If you owe the bank $100 that's your problem. If you owe the bank $100 million, that's the bank's problem." - J. Paul Getty Nvidia may well be at the mercy of them! Hence the recent circular dealing

bee_rider 1y ago

The past/present company they remind me of the most is semiconductor fabs. Significant generation-to-generation R&D investment, significant hardware and infrastructure investment, quite winner-takes-all on the high end, obsoleted in a couple years at most. The main differences are these models are early in their development curve so the jumps are much bigger, and they are entirely digital so they get “shipped” much faster, and open weights seem to be possible. None of those factors seem to make it a more attractive business to be in.

EasyMark 1y ago

In the end Revenues > Costs or you have an issue. That "startup" money will eventually be gone, and you're back to MIMO Money In vs Money Out and if it's not > , you will go bankrupt.

01100011 1y ago

The one thing smaller companies might have is allocated power budgets from power companies. Part of the mad dash to build datacenters right now is just to claim the power so your competitors can't. Now I do think the established players hold an edge here, but I don't think OpenAI/Anthropic/etc are without some bargaining power(hah).