4 ms·

The funny thing about that statement is that if it actually does become true, all of those VCs (and Altman himself), whose job is ostensibly to find the optimal

The funny thing about that statement is that if it actually does become true, all of those VCs (and Altman himself), whose job is ostensibly to find the optimal uses for capital, would immediately become obsolete. Heck, the whole idea that capitalism could just continue along in its current form if true AGI existed is pretty laughable.

benterix 2y ago
There are so many things wrong in this statement (starting with "immediately"). Let's assume they built a system they claim is AGI. Let's assume its energy consumption is smaller than that of a small country. Let's assume that we can verify it's AGI. Let's assume its intelligence is higher than average human. That's many "ifs" and I omitted quite a few. Now, the question is: would you trust it? As a human, a manager, a president? With the current generation, I treat is as a dumb but quick typist, it can create code much faster than I can, but the responsibility to verify it all is entirely on me. We would need decades of proof such an AGI is reliable in order to actually start trusting it - and even then, I'm not sure how safe it would be.
- xvector 2y ago
  If anyone in the thread has used o1 or the real-time voice tools, it's pretty clear AGI is here already, so we are really talking about ASI. You have no option but to trust an ASI as it is all-powerful by definition. If you don't trust ASI, your only option is to prevent it from existing to begin with. Edit: please note that AGI ≠ "human intelligence," just a general intelligence (that may exceed humans in some areas and fall behind in others.)
  - flunhat 2y ago
    > please note that AGI ≠ "human intelligence," just a general intelligence (that may exceed humans in some areas and fall behind in others.) By this definition a calculator would be an AGI. (Behold -- a man!)
    - xvector 2y ago
      Meh, what I've seen is that we continually move the goalposts for AGI, and even GTP-3.5 would have been considered AGI by our standards from just 5 years ago. But if I can't convince you, maybe Norvig can: https://www.noemamag.com/artificial-general-intelligence-is-already-here/ https://www.noemamag.com/artificial-general-intelligence-is-...
      - deleted 2y ago
        [deleted]
  - yunwal 2y ago
    Let’s say you gave o1 an API to control a good robot. Could it throw a football? Could it accomplish even the most basic tasks? If not, it’s not generally intelligent.
    - xvector 2y ago
      > Let’s say you gave o1 an API to control a good robot. Could it throw a football? Maybe. > Could it accomplish even the most basic tasks? Definitely: https://youtu.be/Sq1QZB5baNw https://youtu.be/Sq1QZB5baNw
      - yunwal 2y ago
        I stand corrected
  - benterix 2y ago
    > If you don't trust ASI, your only option is to prevent it from existing to begin with. I don't understand this sentence. I don't trust Generative AI because it often spits out false, inaccurate or made up answers, but I don't believe my "only option is to prevent it from existing".
    - xvector 2y ago
      ASI by definition will be all-powerful or close to it. What are you gonna do if it comes into existence? Because if you don't trust it, you're fucked: - Your attempts to limit its influence on your life will be effective only if the ASI decides to willfully ignore or not notice you. - If it lies or not, there's nothing you can really do. The outcome it wants is practically guaranteed regardless. It will be running circles around your mental capacity like a human drawing circles around an ant on a piece of paper. So what option do you really have but to trust it? I mean, sure, you can not trust the all-powerful god, but your lack of trust will never really have any effect on the real world, or even your life, as the ASI will always get what it wants. Really, your only option is to prevent it from happening to begin with. All that said - I think ASI will be great and people's concerns are overblown.
      - benterix 2y ago
        > ASI by definition will be all-powerful or close to it. That's a huge assumption. AI in SF: all-powerful all-controlling abstract entity. AI in reality: LLMs on thousands of servers with GPUs, operated and controlled by engineers, gated by a startup masquerading as a non-profit and operating for years at a loss, with a web interface and an API, fragile enough that it becomes unavailable at times.
        xvector 2y ago
        You are conflating AGI (what we have today) with ASI (what you are referring to in science-fiction.) These are completely different things and my comment refers to ASI, not AGI.
- BarryMilo 2y ago
  Would you settle for a few days of testing in perfect conditions? Just kidding, companies don't care!
- beAbU 2y ago
  /All Gifts, Bestowed/ by Gayou is a great read that explores this topic.
throwup238 2y ago
The problems with central planning that capitalism ostensibly solves don't exist because of a lack of intelligence, but due to the impedance mismatch between the planner and the people. Making the central planner an AGI would just make it worse, because there's no guarantee that just because it's (super)intelligent it can empathize with its wards and optimize for their needs and desires.
- baq 2y ago
  The problem isn't AGI becomes the oracle central planner. The problem is AGI becomes the central planner, the government and everybody else who currently has a job.
  - visarga 2y ago
    I think there won't be just one AGI, no central planner. LLM abilities leak, other models can catch up in a few months.
- jiggawatts 2y ago
  The problem with all such criticisms is that there is an implicit assumption that humans can be trusted.
  - stoperaticless 2y ago
    I know human limitations. I don’t know AGI limtations. Most humans can not lie all the time. Their true intentions do come out from time to time. AGI might not have that problem - AGI might hide its true intentions for hundreds of years.
    - jiggawatts 2y ago
      Have you… met humans?
- brendoelfrendo 2y ago
  I don't think the concern is that an AGI would become a central planner, but that an AGI would be so much better than human investors that the entire VC class would be outclassed, and that the free market would shift towards using AGI to make investment/capital allocation decisions. Which, of course, runs the risk of turning the whole system into a paperclip optimizing machine that consumes the planet in pursuit of profit; but the VC class seems to desire that anyway, so I don't think we can assume that a free market would consider that a bad outcome.

benterix 2y ago

There are so many things wrong in this statement (starting with "immediately"). Let's assume they built a system they claim is AGI. Let's assume its energy consumption is smaller than that of a small country. Let's assume that we can verify it's AGI. Let's assume its intelligence is higher than average human. That's many "ifs" and I omitted quite a few. Now, the question is: would you trust it? As a human, a manager, a president? With the current generation, I treat is as a dumb but quick typist, it can create code much faster than I can, but the responsibility to verify it all is entirely on me. We would need decades of proof such an AGI is reliable in order to actually start trusting it - and even then, I'm not sure how safe it would be.

xvector 2y ago
If anyone in the thread has used o1 or the real-time voice tools, it's pretty clear AGI is here already, so we are really talking about ASI. You have no option but to trust an ASI as it is all-powerful by definition. If you don't trust ASI, your only option is to prevent it from existing to begin with. Edit: please note that AGI ≠ "human intelligence," just a general intelligence (that may exceed humans in some areas and fall behind in others.)
- flunhat 2y ago
  > please note that AGI ≠ "human intelligence," just a general intelligence (that may exceed humans in some areas and fall behind in others.) By this definition a calculator would be an AGI. (Behold -- a man!)
  - xvector 2y ago
    Meh, what I've seen is that we continually move the goalposts for AGI, and even GTP-3.5 would have been considered AGI by our standards from just 5 years ago. But if I can't convince you, maybe Norvig can: https://www.noemamag.com/artificial-general-intelligence-is-already-here/ https://www.noemamag.com/artificial-general-intelligence-is-...
    - deleted 2y ago
      [deleted]
- yunwal 2y ago
  Let’s say you gave o1 an API to control a good robot. Could it throw a football? Could it accomplish even the most basic tasks? If not, it’s not generally intelligent.
  - xvector 2y ago
    > Let’s say you gave o1 an API to control a good robot. Could it throw a football? Maybe. > Could it accomplish even the most basic tasks? Definitely: https://youtu.be/Sq1QZB5baNw https://youtu.be/Sq1QZB5baNw
    - yunwal 2y ago
      I stand corrected
- benterix 2y ago
  > If you don't trust ASI, your only option is to prevent it from existing to begin with. I don't understand this sentence. I don't trust Generative AI because it often spits out false, inaccurate or made up answers, but I don't believe my "only option is to prevent it from existing".
  - xvector 2y ago
    ASI by definition will be all-powerful or close to it. What are you gonna do if it comes into existence? Because if you don't trust it, you're fucked: - Your attempts to limit its influence on your life will be effective only if the ASI decides to willfully ignore or not notice you. - If it lies or not, there's nothing you can really do. The outcome it wants is practically guaranteed regardless. It will be running circles around your mental capacity like a human drawing circles around an ant on a piece of paper. So what option do you really have but to trust it? I mean, sure, you can not trust the all-powerful god, but your lack of trust will never really have any effect on the real world, or even your life, as the ASI will always get what it wants. Really, your only option is to prevent it from happening to begin with. All that said - I think ASI will be great and people's concerns are overblown.
    - benterix 2y ago
      > ASI by definition will be all-powerful or close to it. That's a huge assumption. AI in SF: all-powerful all-controlling abstract entity. AI in reality: LLMs on thousands of servers with GPUs, operated and controlled by engineers, gated by a startup masquerading as a non-profit and operating for years at a loss, with a web interface and an API, fragile enough that it becomes unavailable at times.
      - xvector 2y ago
        You are conflating AGI (what we have today) with ASI (what you are referring to in science-fiction.) These are completely different things and my comment refers to ASI, not AGI.
BarryMilo 2y ago
Would you settle for a few days of testing in perfect conditions? Just kidding, companies don't care!
beAbU 2y ago
/All Gifts, Bestowed/ by Gayou is a great read that explores this topic.

throwup238 2y ago

The problems with central planning that capitalism ostensibly solves don't exist because of a lack of intelligence, but due to the impedance mismatch between the planner and the people. Making the central planner an AGI would just make it worse, because there's no guarantee that just because it's (super)intelligent it can empathize with its wards and optimize for their needs and desires.

baq 2y ago
The problem isn't AGI becomes the oracle central planner. The problem is AGI becomes the central planner, the government and everybody else who currently has a job.
- visarga 2y ago
  I think there won't be just one AGI, no central planner. LLM abilities leak, other models can catch up in a few months.
jiggawatts 2y ago
The problem with all such criticisms is that there is an implicit assumption that humans can be trusted.
- stoperaticless 2y ago
  I know human limitations. I don’t know AGI limtations. Most humans can not lie all the time. Their true intentions do come out from time to time. AGI might not have that problem - AGI might hide its true intentions for hundreds of years.
  - jiggawatts 2y ago
    Have you… met humans?
brendoelfrendo 2y ago
I don't think the concern is that an AGI would become a central planner, but that an AGI would be so much better than human investors that the entire VC class would be outclassed, and that the free market would shift towards using AGI to make investment/capital allocation decisions. Which, of course, runs the risk of turning the whole system into a paperclip optimizing machine that consumes the planet in pursuit of profit; but the VC class seems to desire that anyway, so I don't think we can assume that a free market would consider that a bad outcome.