12 ms·

What you're talking about is something in the vein of exponential super intelligence. Realistically what actually ends up happening imo, we get human level AGI

What you're talking about is something in the vein of exponential super intelligence. Realistically what actually ends up happening imo, we get human level AGI and hit a ceiling there. Agents replace large portions of the current service economy greatly increasing automation / efficiency for companies. People continue to live their lives, as the idea of having a human level AGI personal assistant becomes normalized and then taken for granted.

brigadier132 2y ago
I think you underestimate what can be accomplished with human level agi. Human level agi could mean 1 million Von Neumann level intelligences cranking 24/7 on humanity's problems.
- lawrenceyan 2y ago
  Yeah, I think you're probably right after further consideration and trying some new models today.
- mentos 2y ago
  Or cranking on super intelligence. What’s the minimum coefficient of human intelligence necessary to boot strap to infinity?
  - scarmig 2y ago
    Infinity intelligence is a very vague and probably ill-defined concept; to go to an impossible extreme, if you're capable of modeling and predicting everything in the universe perfectly at zero cost, what would it even mean to be more intelligent? That is a hard limit on intelligence, but neural networks can't even reach that. What is the actual limit? No one knows. Maybe it's something relatively close to that, modulo physical constraints. Maybe it's right above the maximum human intelligence (and evolution managed to converge to a near optimal architecture). No one knows.
    - astrange 2y ago
      > if you're capable of modeling and predicting everything in the universe perfectly at zero cost As far as we know, it's impossible to model any part of the universe perfectly, although it usually doesn't matter. https://en.wikipedia.org/wiki/No-cloning_theorem https://en.wikipedia.org/wiki/No-cloning_theorem
- TaylorAlexander 2y ago
  Only if there are no hardware limits, which seems highly unlikely.
  - unyttigfjelltol 2y ago
    Right, the comments are assuming an entrepreneur could conjure an army of brains out of nothing. In reality, the question is whether those brains are so much cheaper they open avenues currently unavailable. Would it be cheaper to hire an AGI or a human intern?
- vkou 2y ago
  The biggest problem that humanity has from the perspective of the people with the capital necessary to deploy this is 'How to consolidate more wealth and power into their hands.' One million Von Neumanns working on that 'problem' is not something I'm looking forward to.
andyjohnson0 2y ago
> Agents replace large portions of the current service economy greatly increasing automation / efficiency for companies. > People continue to live their lives Presumably large numbers of those people no longer have jobs, and therefore no income. > we get human level AGI and hit a ceiling there Recently I've been wondering if our best chance for a brake on runaway non-hard-takeoff superintelligence would be that the economy would be trashed.
- lawrenceyan 2y ago
  People will move from the service economy to the entertainment economy powered by Youtube, Tiktok, Mr. Beast, and others. Half-joking. In seriousness, something like a UBI will most likely happen.
  - astrange 2y ago
    The point of UBI or other welfare systems is for people we don't /want/ working, children and the elderly. It's impossible to run out of work for people who are capable of it. As an example, if you have two people and a piece of paper, just tear up the paper into strips, call them money and start exchanging them. Congrats, you both have income now. (This is assuming the AGI has solved the problem of food and stuff, otherwise they're going to have to trade for that and may run out of currency.)
dayvid 2y ago
If we can get human level AGI we will definitely get exponential super intelligence
- trashtester 2y ago
  Every sigmoid ends somewhere. ASI will have limits, but the limits are surely so far beyond human level that it may as well be exponential from our point of view.
dyauspitr 2y ago
If you can get a truly human level AGI, with enough resources it will get to super intelligence by itself.
khafra 2y ago
Looking back at the history of artificial cognition machines, how many of them hit a ceiling at human-level capabilities? - A simple calculator can beat any human at arithmetic - A word processor with a hard disk is orders of magnitude better than any human at memorizing text - No human can approach the ELO of a chess program running on a cheap laptop - No human polymath has journeyman-level knowledge of a tiny fraction of the fields that LLMs can recite textbook answers from Why should we expect AGI to be the first cognitive capabilities that does not quickly shoot past human-level ability?
- lawrenceyan 2y ago
  Yeah so, I just tried a new experimental LLM today. Changed my mind. Think you’re right. At the very least, these models will reach polymath comprehension in every field that exists. And PhD level expertise in every field all at once is by definition superhuman, since people currently are time constrained by limited lifespans.

brigadier132 2y ago

I think you underestimate what can be accomplished with human level agi. Human level agi could mean 1 million Von Neumann level intelligences cranking 24/7 on humanity's problems.

lawrenceyan 2y ago
Yeah, I think you're probably right after further consideration and trying some new models today.
mentos 2y ago
Or cranking on super intelligence. What’s the minimum coefficient of human intelligence necessary to boot strap to infinity?
- scarmig 2y ago
  Infinity intelligence is a very vague and probably ill-defined concept; to go to an impossible extreme, if you're capable of modeling and predicting everything in the universe perfectly at zero cost, what would it even mean to be more intelligent? That is a hard limit on intelligence, but neural networks can't even reach that. What is the actual limit? No one knows. Maybe it's something relatively close to that, modulo physical constraints. Maybe it's right above the maximum human intelligence (and evolution managed to converge to a near optimal architecture). No one knows.
  - astrange 2y ago
    > if you're capable of modeling and predicting everything in the universe perfectly at zero cost As far as we know, it's impossible to model any part of the universe perfectly, although it usually doesn't matter. https://en.wikipedia.org/wiki/No-cloning_theorem https://en.wikipedia.org/wiki/No-cloning_theorem
TaylorAlexander 2y ago
Only if there are no hardware limits, which seems highly unlikely.
- unyttigfjelltol 2y ago
  Right, the comments are assuming an entrepreneur could conjure an army of brains out of nothing. In reality, the question is whether those brains are so much cheaper they open avenues currently unavailable. Would it be cheaper to hire an AGI or a human intern?
vkou 2y ago
The biggest problem that humanity has from the perspective of the people with the capital necessary to deploy this is 'How to consolidate more wealth and power into their hands.' One million Von Neumanns working on that 'problem' is not something I'm looking forward to.

andyjohnson0 2y ago

> Agents replace large portions of the current service economy greatly increasing automation / efficiency for companies. > People continue to live their lives Presumably large numbers of those people no longer have jobs, and therefore no income. > we get human level AGI and hit a ceiling there Recently I've been wondering if our best chance for a brake on runaway non-hard-takeoff superintelligence would be that the economy would be trashed.

lawrenceyan 2y ago
People will move from the service economy to the entertainment economy powered by Youtube, Tiktok, Mr. Beast, and others. Half-joking. In seriousness, something like a UBI will most likely happen.
- astrange 2y ago
  The point of UBI or other welfare systems is for people we don't /want/ working, children and the elderly. It's impossible to run out of work for people who are capable of it. As an example, if you have two people and a piece of paper, just tear up the paper into strips, call them money and start exchanging them. Congrats, you both have income now. (This is assuming the AGI has solved the problem of food and stuff, otherwise they're going to have to trade for that and may run out of currency.)

dayvid 2y ago

If we can get human level AGI we will definitely get exponential super intelligence

trashtester 2y ago
Every sigmoid ends somewhere. ASI will have limits, but the limits are surely so far beyond human level that it may as well be exponential from our point of view.

dyauspitr 2y ago

If you can get a truly human level AGI, with enough resources it will get to super intelligence by itself.

khafra 2y ago

Looking back at the history of artificial cognition machines, how many of them hit a ceiling at human-level capabilities? - A simple calculator can beat any human at arithmetic - A word processor with a hard disk is orders of magnitude better than any human at memorizing text - No human can approach the ELO of a chess program running on a cheap laptop - No human polymath has journeyman-level knowledge of a tiny fraction of the fields that LLMs can recite textbook answers from Why should we expect AGI to be the first cognitive capabilities that does not quickly shoot past human-level ability?

lawrenceyan 2y ago
Yeah so, I just tried a new experimental LLM today. Changed my mind. Think you’re right. At the very least, these models will reach polymath comprehension in every field that exists. And PhD level expertise in every field all at once is by definition superhuman, since people currently are time constrained by limited lifespans.