6 ms·

I have been having this conversation more and more with friends. As a research topic, modern AI is a miracle, and I absolutely love learning about it. As an eco

by peterlk 8mo ago

I have been having this conversation more and more with friends. As a research topic, modern AI is a miracle, and I absolutely love learning about it. As an economic endeavor, it just feels insane. How many hospitals, roads, houses, machine shops, biomanufacturing facilities, parks, forests, laboratories, etc. could we build with the money we’re spending on pretraining models that we throw away next quarter?

uejfiweun 8mo ago
There is a certain logic to it though. If the scaling approaches DO get us to AGI, that's basically going to change everything, forever. And if you assume this is the case, then "our side" has to get there before our geopolitical adversaries do. Because in the long run the expected "hit" from a hostile nation developing AGI and using it to bully "our side" probably really dwarfs the "hit" we take from not developing the infrastructure you mentioned.
- mylifeandtimes 8mo ago
  Here's hoping you are chinese, then.
  - thesmtsolver2 8mo ago
    Why?
  - uejfiweun 8mo ago
    Well, I tried to specifically frame it in a neutral way, to outline the thinking that pretty much all the major nations / companies currently have on this topic.
- A_D_E_P_T 8mo ago
  Any serious LLM user will tell you that there's no way to get from LLM to AGI. These models are vast and, in many ways, clearly superhuman. But they can't venture outside their training data, not even if you hold their hand and guide them. Try getting Suno to write a song in a new genre. Even if you tell it EXACTLY what you want, and provide it with clear examples, it won't be able to do it. This is also why there have been zero-to-very-few new scientific discoveries made by LLM.
  - uejfiweun 8mo ago
    I mean yeah, but that's why there are far more research avenues these days than just pure LLMs, for instance world models. The thinking is that if LLMs can achieve near-human performance in the language domain then we must be very close to achieving human performance in the "general" domain - that's the main thesis of the current AI financial bubble (see articles like AI 2027). And if that is the case, you still want as much compute as possible, both to accelerate research and to achieve greater performance on other architectures that benefit from scaling.
    - pixl97 8mo ago
      The other thing here is we know the human brain learns on far less samples than LLMs in their current form. If there is any kind of learning breakthrough then the amount of compute used for learning could explode overnight
    - rishabhaiover 8mo ago
      How does scaling compute does not go hand-in-hand with energy generation? To me, scaling one and not the other puts a different set of constraints on overall growth. And the energy industry works at a different pace than these hyperscalars scaling compute.
  - pixl97 8mo ago
    Can most people venture outside their training data?
    - nosianu 8mo ago
      Are you seriously comparing chips running AI models and human brains now??? Last time I checked the chips are not rewiring themselves like the brain does, nor does even the software rewrite itself, or the model recalibrate itself - anything that could be called "learning", normal daily work for a human brain. Also, the models are not models of the world, but of our text communication only. Human brains start by building a model of the physical world, from age zero. Much later, on top of that foundation, more abstract ideas emerge, including language. Text, even later. And all of it on a deep layer of a physical world model. The LLM has none of that! It has zero depth behind the words it learned. It's like a human learning some strange symbols and the rules governing their appearance. The human will be able to reproduce valid chains of symbols following the learned rules, but they will never have any understanding of those symbols. In the human case, somebody would have to connect those symbols to their world model by telling them the "meaning" in a way they can already use. For the LLM that is not possible, since it doesn't habe such a model to begin with. How anyone can even entertain the idea of "AGI" based on uncomprehending symbol manipulation, where every symbol has zero depth of a physical world model, only connections to other symbols, is beyond me TBH.
- samrus 8mo ago
  Scaling alone wont get us to AGI. We are in the latter half of this AI summer where the real research has slowed down and even stopped and the MBAs and moguls are doing stupid things For us to take the next step towards AGI, we need an AI winter to hit and the next AI summer to start, the first half of which will produce the advancement we actually need
Kon5ole 8mo ago
I have to admit I'm flip-flopping on the topic, back and forth from skeptic to scared enthusiast. I just made a LLM recreate a decent approximation of the file system browser from the movie Hackers (similar to the SGI one from Jurassic park) in about 10 minutes. At work I've had it do useful features and bug fixes daily for a solid week. Something happened around newyears 2026. The clients, the skills, the mcps, the tools and models reached some new level of usefulness. Or maybe I've been lucky for a week. If it can do things like what I saw last week reliably, then every tool, widget, utility and library currently making money for a single dev or small team of devs is about to get eaten. Maybe even applications like jira, slack, or even salesforce or SAP can be made in-house by even small companies. "Make me a basic CRM". Just a few months ago I found it mostly frustrating to use LLM's and I thought the whole thing was little more than a slight improvement over googling info for myself. But the past week has been mind-blowing. Is it the beginning of the star trek ship computer? If so, it is as big as the smartphone, the internet, or even the invention of the microchip. And then the investments make sense in a way. The problem might end up being that the value created by LLMs will have no customers when everyone is unemployed.
- bojan 8mo ago
  I agree with you, and share the experience. Something changed recently for me as well, where I found the mode to actually get value from these things. I find it refreshing that I don't have to write boilerplate myself or think about the exact syntax of the framework I use. I get to think about the part that adds value. I also have the same experience where we rejected a SAP offering with the idea to build the same thing in-house. But... aside from the obvious fact that building a thing is easier than using and maintaining the thing, the question arose if we even need what SAP offered, or if we get agents to do it. In your example, do you actually need that simple CRM or maybe you can get agents to do the thing without any other additional software? I don't know what this means for our jobs. I do know that, if making software becomes so trivial for everyone, companies will have to find another way to differentiate and compete. And hopefully that's where knowledge workers come in again.
  - r_lee 8mo ago
mike_hearn 8mo ago
FWIW the models aren't thrown away. The weights are used to preinit the next foundation model training run. It helps to reuse weights rather than randomize them even if the model has a somewhat different architecture. As for the rest, constraint on hospital capacity (at least in some countries, not sure about the USA) isn't money for capex, it's doctors unions that restrict training slots.
qaq 8mo ago
Not many. Money is not a perfect abstraction. The raw materials used to produce 100B worth of Nvidia chips will not yield you many hospitals. AI researcher with 100M singup bonus from Meta ain't gonna lay you much brick.
- thwarted 8mo ago
  It's not about the consumption of raw materials or repurposing of the raw materials used for chips. peterlk said: > How many hospitals, roads, houses, machine shops, biomanufacturing facilities, parks, forests, laboratories, etc. could we build with the money we’re spending on pretraining models that we throw away next quarter? It's about using the money for to build things that we actually need and that have more long term utility. No one expects someone with a 100M signing bonus at Meta to lay bricks, but that 100M could be used to buy a lot of bricks and pay a lot of brick layers to build hospitals.
  - OGEnthusiast 8mo ago
    Seems like the main issue is that taxes in America are far too low.
    - qaq 8mo ago
      Again people confuse paper wealth and material assets. If you take half of money of 0.001% people imagine there will be material change in world of atoms but thats not true. You can't take 8 mil Richard Mille watch and build an apartment building. We are mostly resource constrained. There are no material assets to convert all the paper wealth into. Telsa's physical assets are like 5% of Tesla's market cap the rest is cultish belief in Elon. You can't convert that into a hospital. It's trivial to observe on AI side there is unlimited amount of $ available and yet companies are supplied constrained on the atoms side from gas turbines having 3-4 year lead times to ASML running 24/7 prod cycle and yet unable to meet demand.
      - ponector 8mo ago
        You can tax wealth, assets and paper wealth as well. Some countries like Switzerland does it. Annual tax is 0.05-0.3% and that what should billionaires pay to the society.
johnvanommen 8mo ago
> How many hospitals, roads, houses, machine shops, biomanufacturing facilities, parks, forests, laboratories, etc. could we build “We?” This isn’t “our” money. If you buy shares, you get a voice.
YZF 8mo ago
It's not a zero sum game. We could build hospitals and data centers. The reason we are not building hospitals or parks or machine shops have nothing to do with AI. We weren't building them 2 years ago either.
- eviks 8mo ago
  Indeed not zero sum, but a negative sum game to waste money instead of even doing nothing, let alone building something useful.
lII1lIlI11ll 8mo ago
>As a research topic, modern AI is a miracle, and I absolutely love learning about it. As an economic endeavor, it just feels insane. How many hospitals, roads, houses, machine shops, biomanufacturing facilities, parks, forests, laboratories, etc. could we build with the money we’re spending on pretraining models that we throw away next quarter? This is a wrong way to look at it. The right way is to consider that AI investments generate (taxable) economic activity that your government can use to build "hospitals, roads, houses, machine shops, biomanufacturing facilities, parks, forests, laboratories".
- anon7000 8mo ago
  Not so much when there’s a race to the bottom for which municipalities, and states can offer the most tax breaks.
  - lII1lIlI11ll 8mo ago
    People working on those facilities still pay income tax to the municipality no matter how much of a discount the business gets. People buying AI tokes/subscriptions pay VAT to the municipality where they reside.
- efficax 8mo ago
  they pay very little tax and most of the cash is going into datacenters and electricity which provide very little long term employment. llms can do some amazing things but at the same time they’re setting mountains of cash on fire to nudify random women on twitter and generate more spam than we could’ve ever imagined possible
polski-g 8mo ago
Google has zero expected build outs of "forests". They've never mentioned this in their 10k ever. There is no misallocation of Google's money from "forests" to datacenters.

by peterlk 8mo ago

uejfiweun 8mo ago

There is a certain logic to it though. If the scaling approaches DO get us to AGI, that's basically going to change everything, forever. And if you assume this is the case, then "our side" has to get there before our geopolitical adversaries do. Because in the long run the expected "hit" from a hostile nation developing AGI and using it to bully "our side" probably really dwarfs the "hit" we take from not developing the infrastructure you mentioned.

mylifeandtimes 8mo ago
Here's hoping you are chinese, then.
- thesmtsolver2 8mo ago
  Why?
- uejfiweun 8mo ago
  Well, I tried to specifically frame it in a neutral way, to outline the thinking that pretty much all the major nations / companies currently have on this topic.
A_D_E_P_T 8mo ago
Any serious LLM user will tell you that there's no way to get from LLM to AGI. These models are vast and, in many ways, clearly superhuman. But they can't venture outside their training data, not even if you hold their hand and guide them. Try getting Suno to write a song in a new genre. Even if you tell it EXACTLY what you want, and provide it with clear examples, it won't be able to do it. This is also why there have been zero-to-very-few new scientific discoveries made by LLM.
- uejfiweun 8mo ago
  I mean yeah, but that's why there are far more research avenues these days than just pure LLMs, for instance world models. The thinking is that if LLMs can achieve near-human performance in the language domain then we must be very close to achieving human performance in the "general" domain - that's the main thesis of the current AI financial bubble (see articles like AI 2027). And if that is the case, you still want as much compute as possible, both to accelerate research and to achieve greater performance on other architectures that benefit from scaling.
  - pixl97 8mo ago
    The other thing here is we know the human brain learns on far less samples than LLMs in their current form. If there is any kind of learning breakthrough then the amount of compute used for learning could explode overnight
  - rishabhaiover 8mo ago
    How does scaling compute does not go hand-in-hand with energy generation? To me, scaling one and not the other puts a different set of constraints on overall growth. And the energy industry works at a different pace than these hyperscalars scaling compute.
- pixl97 8mo ago
  Can most people venture outside their training data?
  - nosianu 8mo ago
    Are you seriously comparing chips running AI models and human brains now??? Last time I checked the chips are not rewiring themselves like the brain does, nor does even the software rewrite itself, or the model recalibrate itself - anything that could be called "learning", normal daily work for a human brain. Also, the models are not models of the world, but of our text communication only. Human brains start by building a model of the physical world, from age zero. Much later, on top of that foundation, more abstract ideas emerge, including language. Text, even later. And all of it on a deep layer of a physical world model. The LLM has none of that! It has zero depth behind the words it learned. It's like a human learning some strange symbols and the rules governing their appearance. The human will be able to reproduce valid chains of symbols following the learned rules, but they will never have any understanding of those symbols. In the human case, somebody would have to connect those symbols to their world model by telling them the "meaning" in a way they can already use. For the LLM that is not possible, since it doesn't habe such a model to begin with. How anyone can even entertain the idea of "AGI" based on uncomprehending symbol manipulation, where every symbol has zero depth of a physical world model, only connections to other symbols, is beyond me TBH.
samrus 8mo ago
Scaling alone wont get us to AGI. We are in the latter half of this AI summer where the real research has slowed down and even stopped and the MBAs and moguls are doing stupid things For us to take the next step towards AGI, we need an AI winter to hit and the next AI summer to start, the first half of which will produce the advancement we actually need

Kon5ole 8mo ago

I have to admit I'm flip-flopping on the topic, back and forth from skeptic to scared enthusiast. I just made a LLM recreate a decent approximation of the file system browser from the movie Hackers (similar to the SGI one from Jurassic park) in about 10 minutes. At work I've had it do useful features and bug fixes daily for a solid week. Something happened around newyears 2026. The clients, the skills, the mcps, the tools and models reached some new level of usefulness. Or maybe I've been lucky for a week. If it can do things like what I saw last week reliably, then every tool, widget, utility and library currently making money for a single dev or small team of devs is about to get eaten. Maybe even applications like jira, slack, or even salesforce or SAP can be made in-house by even small companies. "Make me a basic CRM". Just a few months ago I found it mostly frustrating to use LLM's and I thought the whole thing was little more than a slight improvement over googling info for myself. But the past week has been mind-blowing. Is it the beginning of the star trek ship computer? If so, it is as big as the smartphone, the internet, or even the invention of the microchip. And then the investments make sense in a way. The problem might end up being that the value created by LLMs will have no customers when everyone is unemployed.

bojan 8mo ago
I agree with you, and share the experience. Something changed recently for me as well, where I found the mode to actually get value from these things. I find it refreshing that I don't have to write boilerplate myself or think about the exact syntax of the framework I use. I get to think about the part that adds value. I also have the same experience where we rejected a SAP offering with the idea to build the same thing in-house. But... aside from the obvious fact that building a thing is easier than using and maintaining the thing, the question arose if we even need what SAP offered, or if we get agents to do it. In your example, do you actually need that simple CRM or maybe you can get agents to do the thing without any other additional software? I don't know what this means for our jobs. I do know that, if making software becomes so trivial for everyone, companies will have to find another way to differentiate and compete. And hopefully that's where knowledge workers come in again.
- r_lee 8mo ago

mike_hearn 8mo ago

FWIW the models aren't thrown away. The weights are used to preinit the next foundation model training run. It helps to reuse weights rather than randomize them even if the model has a somewhat different architecture. As for the rest, constraint on hospital capacity (at least in some countries, not sure about the USA) isn't money for capex, it's doctors unions that restrict training slots.

qaq 8mo ago

Not many. Money is not a perfect abstraction. The raw materials used to produce 100B worth of Nvidia chips will not yield you many hospitals. AI researcher with 100M singup bonus from Meta ain't gonna lay you much brick.

thwarted 8mo ago
It's not about the consumption of raw materials or repurposing of the raw materials used for chips. peterlk said: > How many hospitals, roads, houses, machine shops, biomanufacturing facilities, parks, forests, laboratories, etc. could we build with the money we’re spending on pretraining models that we throw away next quarter? It's about using the money for to build things that we actually need and that have more long term utility. No one expects someone with a 100M signing bonus at Meta to lay bricks, but that 100M could be used to buy a lot of bricks and pay a lot of brick layers to build hospitals.
- OGEnthusiast 8mo ago
  Seems like the main issue is that taxes in America are far too low.
  - qaq 8mo ago
    Again people confuse paper wealth and material assets. If you take half of money of 0.001% people imagine there will be material change in world of atoms but thats not true. You can't take 8 mil Richard Mille watch and build an apartment building. We are mostly resource constrained. There are no material assets to convert all the paper wealth into. Telsa's physical assets are like 5% of Tesla's market cap the rest is cultish belief in Elon. You can't convert that into a hospital. It's trivial to observe on AI side there is unlimited amount of $ available and yet companies are supplied constrained on the atoms side from gas turbines having 3-4 year lead times to ASML running 24/7 prod cycle and yet unable to meet demand.
    - ponector 8mo ago
      You can tax wealth, assets and paper wealth as well. Some countries like Switzerland does it. Annual tax is 0.05-0.3% and that what should billionaires pay to the society.

johnvanommen 8mo ago

> How many hospitals, roads, houses, machine shops, biomanufacturing facilities, parks, forests, laboratories, etc. could we build “We?” This isn’t “our” money. If you buy shares, you get a voice.

YZF 8mo ago

It's not a zero sum game. We could build hospitals and data centers. The reason we are not building hospitals or parks or machine shops have nothing to do with AI. We weren't building them 2 years ago either.

eviks 8mo ago
Indeed not zero sum, but a negative sum game to waste money instead of even doing nothing, let alone building something useful.

lII1lIlI11ll 8mo ago

>As a research topic, modern AI is a miracle, and I absolutely love learning about it. As an economic endeavor, it just feels insane. How many hospitals, roads, houses, machine shops, biomanufacturing facilities, parks, forests, laboratories, etc. could we build with the money we’re spending on pretraining models that we throw away next quarter? This is a wrong way to look at it. The right way is to consider that AI investments generate (taxable) economic activity that your government can use to build "hospitals, roads, houses, machine shops, biomanufacturing facilities, parks, forests, laboratories".

anon7000 8mo ago
Not so much when there’s a race to the bottom for which municipalities, and states can offer the most tax breaks.
- lII1lIlI11ll 8mo ago
  People working on those facilities still pay income tax to the municipality no matter how much of a discount the business gets. People buying AI tokes/subscriptions pay VAT to the municipality where they reside.
efficax 8mo ago
they pay very little tax and most of the cash is going into datacenters and electricity which provide very little long term employment. llms can do some amazing things but at the same time they’re setting mountains of cash on fire to nudify random women on twitter and generate more spam than we could’ve ever imagined possible

polski-g 8mo ago

Google has zero expected build outs of "forests". They've never mentioned this in their 10k ever. There is no misallocation of Google's money from "forests" to datacenters.