5 ms·

It's justified if AGI is possible. If AGI is possible, then the entire human economy stops making sense as far as money goes, and 'owning' part of OpenAI gives

It's justified if AGI is possible. If AGI is possible, then the entire human economy stops making sense as far as money goes, and 'owning' part of OpenAI gives you power. That is of course, assuming AGI is possible and exponential, and that marketshare goes to a single entity instead of a set of entities. Lots of big assumptions. Seems like we're heading towards a slow-lackluster singularity though.

deleted 2y ago
[deleted]
philipkglass 2y ago
If AGI is possible, then the entire human economy stops making sense as far as money goes, and 'owning' part of OpenAI gives you power. That's if AGI is possible and not easily replicated. If AGI can be copied and/or re-developed like other software then the value of owning OpenAI stock is more like owning stock in copper producers or other commodity sector companies. (It might even be a poorer investment. Even AGI can't create copper atoms, so owners of real physical resources could be in a better position in a post-human-labor world.)
- whatshisface 2y ago
  This belief comes from confusing the singularity (every atom on Earth is converted into a giant image of Sam Altman) with AGI (a store employee navigates a confrontation with an unruly customer, then goes home and wins at Super Mario).
  - fullstackchris 2y ago
    Exactly. I continually fail to see how "the entire human economy ends" overnight with another human like agent out there - especially if its confined to a server in the first place - it can't even "go home" :)
  - baobabKoodaa 2y ago
    If I recall correctly, these terms were used more or less interchangeably for a few decades, until 2020 or so, when OpenAI started making actual progress towards AGI, and it was clear that the type of AGI that could be imagined at that point, would not be of the type that would produce singularity.
  - Teever 2y ago
    But what if that AGI can fit inside a humanoid robot and that robot is capable of self replication even if it means digging the sand out of the ground to make silicon with a spade?
    - Terr_ 2y ago
      We already have humanoid intelligeces that self-assemble and power from common materials, as a colony of incredibly advanced nanobots.
      - Teever 2y ago
        Yes. The goal is to emulate that with different substrates to understand how it works and to have better control over existing self-replicating systems.
- AnimalMuppet 2y ago
  The GP said, "and exponential". If AGI is exponential, then the first one will have a head start advantage that compounds over time. That is going to be hard to overcome.
  - philipkglass 2y ago
    I believe that AGI cannot be exponential for long because any intelligent agent can only approach nature's limits asymptotically. The first company with AGI will be about as much ahead as, say, the first company with electrical generators [1]. A lot of science fiction about a technological singularity assumes that AGI will discover and apply new physics to develop currently-believed-impossible inventions, but I don't consider that plausible myself. I believe that the discovery of new physics will be intellectually satisfying but generally inapplicable to industry, much like how solving the cosmological lithium problem will be career-defining for whoever does it but won't have any application to lithium batteries. https://en.wikipedia.org/wiki/Cosmological_lithium_problem https://en.wikipedia.org/wiki/Cosmological_lithium_problem [1] https://en.wikipedia.org/wiki/Siemens#1847_to_1901 https://en.wikipedia.org/wiki/Siemens#1847_to_1901
    - datadrivenangel 2y ago
      I don't recall editing my message, but HN can be wonky sometimes. :) Nothing is truly exponential for long, but the logistic curve could be big enough to do almost anything if you get imaginative. Without new physics, there are still some places where we can do some amazing things with the equivalent of several trillion dollars of applied R&D, which AGI gets you.
      - philipkglass 2y ago
        I had to edit my message just now because I was actually unsure if you edited. Sorry for any miscommunication.
      - terribleperson 2y ago
        This depends on what a hypothetical 'AGI' actually costs. If a real AGI is achieved, but it costs more per unit of work than a human does... it won't do anyone much good.
- richardw 2y ago
  The first AGI will have such an advantage. It’ll be the first thing that is smart and tireless, can do anything from continuously hacking enemy networks to trading across all investment classes, to basically taking over the news cycle on social media. It would print money and power.
  - HDThoreaun 2y ago
    Depends on how efficient it is. If it requires more processing power than we have to do all these things competitors will have time to catch up while new hardware is created.
Terr_ 2y ago
One strata in that assumption-heap to call out explicitly: Assuming LLMs are an enabling route to AGI and not a dead-end or supplemental feature.
api 2y ago
If AGI is possible then that too becomes a commodity and we experience a massive round of deflation in the cost of everything not intrinsically rare. Land, food, rare materials, energy, and anything requiring human labor is expensive and everything else is almost free. I don't see how OpenAI wouldn't crash and burn here. Given the history of models it would be at most a year before you'd have open AGI, then the horse is out of the barn and the horse begins to self-improve. Pretty soon the horse is a unicorn, then it's a Satyr, and so on. (I am a near-term AGI skeptic BTW, but I could be wrong.) OpenAI's valuation is a mixture of hype speculation and the "golden boy" cult around Sam Altman. In the latter sense it's similar to the golden boy cults around Elon Musk and (politically) Donald Trump. To some extent these cults work because they are self-fulfilling feedback loops: these people raise tons of capital (economic or political) because everyone knows they're going to raise tons of capital so they raise tons of capital.
parpfish 2y ago
Well, AGI would make the brainy information worker part of the economy obsolete. Well still need the jobs that interact with the physical world for quite a while. So… all us HN types should get ready to work the mines or pick vegetables
- throwup238 2y ago
  If we hit true AGI, physical labor won’t be far behind the knowledge workers. The first thing industrial manufacturers will do is turn it towards designing robotics, automating the design of factories, and researching better electromechanical components like synthetic muscle to replace human dexterity. IMO we’re going to hit the point where AI can work on designing automation to replace physical labor before we hit true AGI, much like we’re seeing with coding.
hdjjhhvvhga 2y ago
> If AGI is possible, then the entire human economy stops making sense as far as money goes I heard people on HN saying this (even without the money condition) and I fail to grasp the reasoning behind it. Suppose in a few years Altman announces a model, say o11, that is supposedly AGI, and in several benchmarks it hits over 90%. I don't believe it's possible with LLMs because of their inherent limitations but let's assume it can solve general tasks in a way similar to an average human. Now, how come that "the entire human economy stops making sense"? In order to eat, we need farmers, we need construction workers, shops etc. As for white collar workers, you will need a whole range of people to maintain and further develop this AGI. So IMHO the opposite is true: the human economy will work exactly as before but the job market will continue to evolve withe people using AGI in a similar way that they use LLMs now but probably with greater confidence. (Or not.)
- datadrivenangel 2y ago
  Why do we work? Ultimately, we work to live.* If the value of our labor is determined by scarcity, then what happens when productivity goes nearly infinite and the scarcity goes away? We still have needs and wants, but the current market will be completely inverted.
- exe34 2y ago
  If you think about all the people trying to automate away farming, construction, transport/delivery - these people doing the automation themselves get automated out first, and the automation figures out how to do the rest. So a fully robotic economy is not far off, if you can achieve AGI.
- SmooL 2y ago
  The thinking goes: - any job that can be done on a computer is immediately outsourced to AI, since the AI is smarter and cheaper than humans - humanoid robots are built that are cheap to produce, using tech advances that the AI discovered - any job that can be done by a human is immediately outsourced to a robot, since the robot is better/faster/stronger/cheaper than humans
UltraSane 2y ago
If AGI is invented and the inventor tries to keep it secret then everyone in the world will be trying to steal it. And funding to independently create it would become effectively unlimited once it has been proven possible, much like with nuclear weapons.
robertlagrant 2y ago
> If AGI is possible, then the entire human economy stops making sense as far as money goes, What does this mean in terms of making me coffee or building houses?
- com2kid 2y ago
  If we can simulate a full human intelligence at a reasonable speed, we can simulate 100 of them and ask the AGI to figure out how to make itself 10x faster. Rinse and repeat. That is exponential take off. At the point where you have an army of AIs running at 1000x human speed it can just ask it to design the mechanisms for and write the code to make robots that automate any possible physical task.
  - GOD_Over_Djinn 2y ago
    This sounds like magic, not science.
    - EMIRELADERO 2y ago
      What do you mean by this? Is there any fundamental property of intelligence, physicality, or the universe, that you think wouldn't let this work?
      - fullstackchris 2y ago
        Not OP but yes. Electron size vs band gap, computing costs (in terms of electricity) any other raw materials needed for that energy, etc... sigh... its physics, always physics... what fundamental property of physics do you think would let a vertical take off in intelligence occur?
        datadrivenangel 2y ago
        If you look at the rate of mathematical operations conducted, we're already going hard vertical. Physics and material limitations will slow that eventually as we reach a marginal return on converting the planet to computer chips, but we're in the singularity as proxy measured by mathematical operations.
deleted 2y ago
[deleted]
Animats 2y ago
We may not need smarter AI. Just less stupid AI. The big problem with LLMs is that most of the time they act smart, and some of the time they do really, really dumb things and don't notice. It's not the ceiling that's the problem. It's the floor. Which is why, as the article points out, "agents" aren't very useful yet. You can't trust them to not screw up big-time.
torginus 2y ago
I was thinking about how the economy has been actively makes less sense and gets divorced more and more from reality year after year, AI or not. It's the simple fact that the ability of assets to generate wealth has far outstripped the abiliy of individuals to earn money by working. Somehow real estate has become so expensive everywhere that owning a shitty apartment is impossible for the vast majority. When the world's population was exploding during the 20th century, housing prices were not a problem, yet somehow nowadays, it's impossible to build affordable housing to bring the prices down, though the population is stagnant or growing slowly. A company can be worth $1B if someone invests $10m in it for 1% stake - where did the remaining $990m come from? Likewise, the stock market is full of trillion-dollar companies whose valuations beggar all explanation, considering the sizes of the markets they are serving. The rich elites are using the wealth to control access to basic human needs (namely housing and healthcare) to squeeze the working population for every drop of money. Every wealth metric shows the 1% and the 1% of the 1% control successively larger portions of the economic pie. At this point money is ceasing to be a proxy for value and is becoming a tool for population control. And the weird thing is it didn't use to be nearly this bad even a decade ago, and we can only guess how bad it will get in a decade, AGI or not. Anyway, I don't want to turn this into a fully-written manifesto, but I have trouble expressing these ideas in a concise manner.
- nyarlathotep_ 2y ago
  > And the weird thing is it didn't use to be nearly this bad even a decade ago, and we can only guess how bad it will get in a decade, AGI or not. The last 5 years have reflected a substantial decline in QOL in the states; you don't even have to to look back that far. The coronacircus money-printing really accelerated the decline.
- ac29 2y ago
  > Somehow real estate has become so expensive everywhere that owning a shitty apartment is impossible for the vast majority. Approximately 2/3s of homes in the US are owner occupied.
  - yen223 2y ago

deleted 2y ago

[deleted]

philipkglass 2y ago

If AGI is possible, then the entire human economy stops making sense as far as money goes, and 'owning' part of OpenAI gives you power. That's if AGI is possible and not easily replicated. If AGI can be copied and/or re-developed like other software then the value of owning OpenAI stock is more like owning stock in copper producers or other commodity sector companies. (It might even be a poorer investment. Even AGI can't create copper atoms, so owners of real physical resources could be in a better position in a post-human-labor world.)

whatshisface 2y ago
This belief comes from confusing the singularity (every atom on Earth is converted into a giant image of Sam Altman) with AGI (a store employee navigates a confrontation with an unruly customer, then goes home and wins at Super Mario).
- fullstackchris 2y ago
  Exactly. I continually fail to see how "the entire human economy ends" overnight with another human like agent out there - especially if its confined to a server in the first place - it can't even "go home" :)
- baobabKoodaa 2y ago
  If I recall correctly, these terms were used more or less interchangeably for a few decades, until 2020 or so, when OpenAI started making actual progress towards AGI, and it was clear that the type of AGI that could be imagined at that point, would not be of the type that would produce singularity.
- Teever 2y ago
  But what if that AGI can fit inside a humanoid robot and that robot is capable of self replication even if it means digging the sand out of the ground to make silicon with a spade?
  - Terr_ 2y ago
    We already have humanoid intelligeces that self-assemble and power from common materials, as a colony of incredibly advanced nanobots.
    - Teever 2y ago
      Yes. The goal is to emulate that with different substrates to understand how it works and to have better control over existing self-replicating systems.
AnimalMuppet 2y ago
The GP said, "and exponential". If AGI is exponential, then the first one will have a head start advantage that compounds over time. That is going to be hard to overcome.
- philipkglass 2y ago
  I believe that AGI cannot be exponential for long because any intelligent agent can only approach nature's limits asymptotically. The first company with AGI will be about as much ahead as, say, the first company with electrical generators [1]. A lot of science fiction about a technological singularity assumes that AGI will discover and apply new physics to develop currently-believed-impossible inventions, but I don't consider that plausible myself. I believe that the discovery of new physics will be intellectually satisfying but generally inapplicable to industry, much like how solving the cosmological lithium problem will be career-defining for whoever does it but won't have any application to lithium batteries. https://en.wikipedia.org/wiki/Cosmological_lithium_problem https://en.wikipedia.org/wiki/Cosmological_lithium_problem [1] https://en.wikipedia.org/wiki/Siemens#1847_to_1901 https://en.wikipedia.org/wiki/Siemens#1847_to_1901
  - datadrivenangel 2y ago
    I don't recall editing my message, but HN can be wonky sometimes. :) Nothing is truly exponential for long, but the logistic curve could be big enough to do almost anything if you get imaginative. Without new physics, there are still some places where we can do some amazing things with the equivalent of several trillion dollars of applied R&D, which AGI gets you.
    - philipkglass 2y ago
      I had to edit my message just now because I was actually unsure if you edited. Sorry for any miscommunication.
    - terribleperson 2y ago
      This depends on what a hypothetical 'AGI' actually costs. If a real AGI is achieved, but it costs more per unit of work than a human does... it won't do anyone much good.
richardw 2y ago
The first AGI will have such an advantage. It’ll be the first thing that is smart and tireless, can do anything from continuously hacking enemy networks to trading across all investment classes, to basically taking over the news cycle on social media. It would print money and power.
- HDThoreaun 2y ago
  Depends on how efficient it is. If it requires more processing power than we have to do all these things competitors will have time to catch up while new hardware is created.

Terr_ 2y ago

One strata in that assumption-heap to call out explicitly: Assuming LLMs are an enabling route to AGI and not a dead-end or supplemental feature.

api 2y ago

If AGI is possible then that too becomes a commodity and we experience a massive round of deflation in the cost of everything not intrinsically rare. Land, food, rare materials, energy, and anything requiring human labor is expensive and everything else is almost free. I don't see how OpenAI wouldn't crash and burn here. Given the history of models it would be at most a year before you'd have open AGI, then the horse is out of the barn and the horse begins to self-improve. Pretty soon the horse is a unicorn, then it's a Satyr, and so on. (I am a near-term AGI skeptic BTW, but I could be wrong.) OpenAI's valuation is a mixture of hype speculation and the "golden boy" cult around Sam Altman. In the latter sense it's similar to the golden boy cults around Elon Musk and (politically) Donald Trump. To some extent these cults work because they are self-fulfilling feedback loops: these people raise tons of capital (economic or political) because everyone knows they're going to raise tons of capital so they raise tons of capital.

parpfish 2y ago

Well, AGI would make the brainy information worker part of the economy obsolete. Well still need the jobs that interact with the physical world for quite a while. So… all us HN types should get ready to work the mines or pick vegetables

throwup238 2y ago
If we hit true AGI, physical labor won’t be far behind the knowledge workers. The first thing industrial manufacturers will do is turn it towards designing robotics, automating the design of factories, and researching better electromechanical components like synthetic muscle to replace human dexterity. IMO we’re going to hit the point where AI can work on designing automation to replace physical labor before we hit true AGI, much like we’re seeing with coding.

hdjjhhvvhga 2y ago

> If AGI is possible, then the entire human economy stops making sense as far as money goes I heard people on HN saying this (even without the money condition) and I fail to grasp the reasoning behind it. Suppose in a few years Altman announces a model, say o11, that is supposedly AGI, and in several benchmarks it hits over 90%. I don't believe it's possible with LLMs because of their inherent limitations but let's assume it can solve general tasks in a way similar to an average human. Now, how come that "the entire human economy stops making sense"? In order to eat, we need farmers, we need construction workers, shops etc. As for white collar workers, you will need a whole range of people to maintain and further develop this AGI. So IMHO the opposite is true: the human economy will work exactly as before but the job market will continue to evolve withe people using AGI in a similar way that they use LLMs now but probably with greater confidence. (Or not.)

datadrivenangel 2y ago
Why do we work? Ultimately, we work to live.* If the value of our labor is determined by scarcity, then what happens when productivity goes nearly infinite and the scarcity goes away? We still have needs and wants, but the current market will be completely inverted.
exe34 2y ago
If you think about all the people trying to automate away farming, construction, transport/delivery - these people doing the automation themselves get automated out first, and the automation figures out how to do the rest. So a fully robotic economy is not far off, if you can achieve AGI.
SmooL 2y ago
The thinking goes: - any job that can be done on a computer is immediately outsourced to AI, since the AI is smarter and cheaper than humans - humanoid robots are built that are cheap to produce, using tech advances that the AI discovered - any job that can be done by a human is immediately outsourced to a robot, since the robot is better/faster/stronger/cheaper than humans

UltraSane 2y ago

If AGI is invented and the inventor tries to keep it secret then everyone in the world will be trying to steal it. And funding to independently create it would become effectively unlimited once it has been proven possible, much like with nuclear weapons.

robertlagrant 2y ago

> If AGI is possible, then the entire human economy stops making sense as far as money goes, What does this mean in terms of making me coffee or building houses?

com2kid 2y ago
If we can simulate a full human intelligence at a reasonable speed, we can simulate 100 of them and ask the AGI to figure out how to make itself 10x faster. Rinse and repeat. That is exponential take off. At the point where you have an army of AIs running at 1000x human speed it can just ask it to design the mechanisms for and write the code to make robots that automate any possible physical task.
- GOD_Over_Djinn 2y ago
  This sounds like magic, not science.
  - EMIRELADERO 2y ago
    What do you mean by this? Is there any fundamental property of intelligence, physicality, or the universe, that you think wouldn't let this work?
    - fullstackchris 2y ago
      Not OP but yes. Electron size vs band gap, computing costs (in terms of electricity) any other raw materials needed for that energy, etc... sigh... its physics, always physics... what fundamental property of physics do you think would let a vertical take off in intelligence occur?
      - datadrivenangel 2y ago
        If you look at the rate of mathematical operations conducted, we're already going hard vertical. Physics and material limitations will slow that eventually as we reach a marginal return on converting the planet to computer chips, but we're in the singularity as proxy measured by mathematical operations.

deleted 2y ago

[deleted]

Animats 2y ago

We may not need smarter AI. Just less stupid AI. The big problem with LLMs is that most of the time they act smart, and some of the time they do really, really dumb things and don't notice. It's not the ceiling that's the problem. It's the floor. Which is why, as the article points out, "agents" aren't very useful yet. You can't trust them to not screw up big-time.

torginus 2y ago

I was thinking about how the economy has been actively makes less sense and gets divorced more and more from reality year after year, AI or not. It's the simple fact that the ability of assets to generate wealth has far outstripped the abiliy of individuals to earn money by working. Somehow real estate has become so expensive everywhere that owning a shitty apartment is impossible for the vast majority. When the world's population was exploding during the 20th century, housing prices were not a problem, yet somehow nowadays, it's impossible to build affordable housing to bring the prices down, though the population is stagnant or growing slowly. A company can be worth $1B if someone invests $10m in it for 1% stake - where did the remaining $990m come from? Likewise, the stock market is full of trillion-dollar companies whose valuations beggar all explanation, considering the sizes of the markets they are serving. The rich elites are using the wealth to control access to basic human needs (namely housing and healthcare) to squeeze the working population for every drop of money. Every wealth metric shows the 1% and the 1% of the 1% control successively larger portions of the economic pie. At this point money is ceasing to be a proxy for value and is becoming a tool for population control. And the weird thing is it didn't use to be nearly this bad even a decade ago, and we can only guess how bad it will get in a decade, AGI or not. Anyway, I don't want to turn this into a fully-written manifesto, but I have trouble expressing these ideas in a concise manner.

nyarlathotep_ 2y ago
> And the weird thing is it didn't use to be nearly this bad even a decade ago, and we can only guess how bad it will get in a decade, AGI or not. The last 5 years have reflected a substantial decline in QOL in the states; you don't even have to to look back that far. The coronacircus money-printing really accelerated the decline.
ac29 2y ago
> Somehow real estate has become so expensive everywhere that owning a shitty apartment is impossible for the vast majority. Approximately 2/3s of homes in the US are owner occupied.
- yen223 2y ago