4 ms·

> ...but I can't understand why there's so much focus on AGI. Lots of software engineers have spent their lives reading sci-fi that features AGI, and they're e

by tivert 3y ago

> ...but I can't understand why there's so much focus on AGI. Lots of software engineers have spent their lives reading sci-fi that features AGI, and they're excited by/lost in that fantasy. It's interesting to see that in people who often view themselves as hyper-rational.

tim333 3y ago
AGI as in computer intelligence that out does humans would be a huge deal in practical terms. Chat GPT and similar are kind of like handy toys. With proper AGI you could link it to a robot body and tell it to go off, design a better version of itself, make a billion more robots and take over the world. It's a different category of thing. And if you think that's just sci-fi I think you'll get a surprise at some point during your life.
- tivert 3y ago
  > With proper AGI you could link it to a robot body and tell it to go off, design a better version of itself, make a billion more robots and take over the world. ^^^^^ literally a plot ripped from the pages of science fiction used to reason about the real world. > And if you think that's just sci-fi I think you'll get a surprise at some point during your life. Such faith that fantasy can be made real. Wake me up when you have my hyperdrive ready.
  - tim333 3y ago
    I've never actually read that in fiction. It's just logical really. I mean I'm sure it is in fiction somewhere because the idea is obvious.
    - tivert 3y ago
      > I've never actually read that in fiction. I find that hard to believe. Ever watch Terminator? But even if true, that science-fictional plot is so pervasive it would be easy to pick up from the millions who have the software engineer's blurry line between fantasy and reality. > It's just logical really. OK, then. You're a GI, go off and build an army of better yous and take over the world.
      - TeMPOraL 3y ago
        The idea is indeed logical and stupidly obvious, once you learn the basics of what "optimization" means, or what "recursion" is. > I find that hard to believe. Ever watched Terminator? Terminator has fuck all to do with recursive self-improvement. Don't confuse people who grew up on sci-fi with people who casually went to see Terminator or some other pop-culture artifact featuring some kind of "AI". > OK, then. You're a GI, go off and build an army of better yous and take over the world. What do you think the drama with eugenics, genetic engineering and designer babies is around? It's literally humans trying to make better humans in the only way that is available - reproduction. AI made in silica would be more malleable, easier and cheaper to replicate. Self-improving software isn't even a fantasy; it exists in many forms - though it's far from open-ended like a self-improving GI would be.
        tivert 3y ago
        >>> I've never actually read that in fiction. >> I find that hard to believe. Ever watched Terminator? > Terminator has fuck all to do with recursive self-improvement. Don't confuse people who grew up on sci-fi with people who casually went to see Terminator or some other pop-culture artifact featuring some kind of "AI". You're not following the thread. The future timeline in Terminator does involve something like an AI making "a billion more robots [to] take over the world." The popularity of that and similar sci-fi makes that claim that someone has never encountered it hard to believe. > What do you think the drama with eugenics, genetic engineering and designer babies is around? So how has that been going? Those things should also probably be labeled "science fiction." > AI made in silica would be more malleable, easier and cheaper to replicate. Self-improving software isn't even a fantasy; it exists in many forms - though it's far from open-ended like a self-improving GI would be. Fantasies based on squishy assumptions. How do you know it would have an easier job optimizing itself than humans have? How do you know there isn't some fundamental contradiction in the concept of "superintelligence" that these fantasies are based on? Or even just some practical resource limits that makes the fantasy impossible?
        TeMPOraL 3y ago
- 8note 3y ago
  So I link this to my Roomba, and in a few days time it'll design and build a much better Roomba, while confined to my apartment with only wheels and a vacuum to actuate? I think a much more realistic outcome is that you put it into a fancier robot body that can go outside, and by the end of the week it's scrap in a homeless camp chop shop
TeMPOraL 3y ago
> It's interesting to see that in people who often view themselves as hyper-rational. It's perhaps because they are rational enough to realize, thanks to the same knowledge/skill that put them on the software engineer careers, that AGI isn't a fantasy but a possibility and a potentially very big deal.

by tivert 3y ago

tim333 3y ago

AGI as in computer intelligence that out does humans would be a huge deal in practical terms. Chat GPT and similar are kind of like handy toys. With proper AGI you could link it to a robot body and tell it to go off, design a better version of itself, make a billion more robots and take over the world. It's a different category of thing. And if you think that's just sci-fi I think you'll get a surprise at some point during your life.

tivert 3y ago
> With proper AGI you could link it to a robot body and tell it to go off, design a better version of itself, make a billion more robots and take over the world. ^^^^^ literally a plot ripped from the pages of science fiction used to reason about the real world. > And if you think that's just sci-fi I think you'll get a surprise at some point during your life. Such faith that fantasy can be made real. Wake me up when you have my hyperdrive ready.
- tim333 3y ago
  I've never actually read that in fiction. It's just logical really. I mean I'm sure it is in fiction somewhere because the idea is obvious.
  - tivert 3y ago
    > I've never actually read that in fiction. I find that hard to believe. Ever watch Terminator? But even if true, that science-fictional plot is so pervasive it would be easy to pick up from the millions who have the software engineer's blurry line between fantasy and reality. > It's just logical really. OK, then. You're a GI, go off and build an army of better yous and take over the world.
    - TeMPOraL 3y ago
      The idea is indeed logical and stupidly obvious, once you learn the basics of what "optimization" means, or what "recursion" is. > I find that hard to believe. Ever watched Terminator? Terminator has fuck all to do with recursive self-improvement. Don't confuse people who grew up on sci-fi with people who casually went to see Terminator or some other pop-culture artifact featuring some kind of "AI". > OK, then. You're a GI, go off and build an army of better yous and take over the world. What do you think the drama with eugenics, genetic engineering and designer babies is around? It's literally humans trying to make better humans in the only way that is available - reproduction. AI made in silica would be more malleable, easier and cheaper to replicate. Self-improving software isn't even a fantasy; it exists in many forms - though it's far from open-ended like a self-improving GI would be.
      - tivert 3y ago
        >>> I've never actually read that in fiction. >> I find that hard to believe. Ever watched Terminator? > Terminator has fuck all to do with recursive self-improvement. Don't confuse people who grew up on sci-fi with people who casually went to see Terminator or some other pop-culture artifact featuring some kind of "AI". You're not following the thread. The future timeline in Terminator does involve something like an AI making "a billion more robots [to] take over the world." The popularity of that and similar sci-fi makes that claim that someone has never encountered it hard to believe. > What do you think the drama with eugenics, genetic engineering and designer babies is around? So how has that been going? Those things should also probably be labeled "science fiction." > AI made in silica would be more malleable, easier and cheaper to replicate. Self-improving software isn't even a fantasy; it exists in many forms - though it's far from open-ended like a self-improving GI would be. Fantasies based on squishy assumptions. How do you know it would have an easier job optimizing itself than humans have? How do you know there isn't some fundamental contradiction in the concept of "superintelligence" that these fantasies are based on? Or even just some practical resource limits that makes the fantasy impossible?
        TeMPOraL 3y ago
8note 3y ago
So I link this to my Roomba, and in a few days time it'll design and build a much better Roomba, while confined to my apartment with only wheels and a vacuum to actuate? I think a much more realistic outcome is that you put it into a fancier robot body that can go outside, and by the end of the week it's scrap in a homeless camp chop shop

TeMPOraL 3y ago

> It's interesting to see that in people who often view themselves as hyper-rational. It's perhaps because they are rational enough to realize, thanks to the same knowledge/skill that put them on the software engineer careers, that AGI isn't a fantasy but a possibility and a potentially very big deal.