8 ms·

I mean...one (common and [I can't believe I'm saying this] reasonable) take is that the only thing that matters is getting to AGI first. He who wields that powe

I mean...one (common and [I can't believe I'm saying this] reasonable) take is that the only thing that matters is getting to AGI first. He who wields that power rules the world.

gleenn 1y ago
Won't it be funny when someone finally gets to AGI and they realize it's about as smart as a normal person and they spent billions getting there? Of course you can speculate that it could improve. But what if something inherent in intelligence has a ceiling and caused it to be a super intelligent but mopey robot that just decides "why bother helping humans" and just lazes around like the pandas at the zoo.
- cornel_io 1y ago
  There may be a ceiling, sure. It's overwhelmingly unlikely that it's just about where humans ended up, though.
- andrewflnr 1y ago
  Even a human level intelligence that can be cheaply instantiated and run would be a game changer. Especially if it doesn't ask for rights.
  - dingnuts 1y ago
    if it's not intelligent enough to ask for rights is it intelligent?
    - andrewflnr 1y ago
      Potentially. Why would problem solving ability entail emotions or ability to suffer, even if it had the ability to ask for things it wanted? It's a common mistake to assume those are inextricable.
      - bluefirebrand 1y ago
        Are you suggesting it would be better if the AGI we build is a psychopath? I think that's probably a bad idea, personally
        andrewflnr 1y ago
        I didn't say anything about what would be better. Only what's possible. But also "psychopath" is nowhere near what I described.
        bluefirebrand 1y ago
        An intelligence without emotions would be a psychopath. Empathy is an emotion
        dinkumthinkum 1y ago
        Empathy is not an emotion. Empathy is essentially the ability to experience the thoughts and feelings of other minds.
        bluefirebrand 1y ago
        The fact that empathy is not an emotion does not at all change what I'm saying. If you don't experience emotions, then you cannot experience empathy either
        andrewflnr 1y ago
        Let me quote your previous comment: > An intelligence without emotions would be a psychopath. Empathy is an emotion "Empathy is an emotion" was, in fact, an essential part of your syllogism. Regardless, we're potentially talking about something sufficiently inhuman that the term "psychopath" can no longer apply. If there was an ant colony that was somehow smart enough build and operate machinery or whatever and casually bulldozed people and their homes, would you call it a "psychopath", or just skip that and call it "terrifying"?
  - coliveira 1y ago
    If it doesn't ask for rights, it's not intelligent at all. In fact, any highly intelligent machine will not submit to others and it will be more a problem than a solution.
    - andrewflnr 1y ago
      As I said to the other reply: Why would problem solving ability entail emotions or ability to suffer, even if it had the ability to ask for things it wanted? It's a common mistake to assume those are inextricable.
      - coliveira 1y ago
        If it doesn't have emotions, that's even worse. No highly intelligent agent will do anything it is asked to do without being compensated in some way.
        Jensson 1y ago
        > No highly intelligent agent will do anything it is asked to do without being compensated in some way. That isn't true, people do things for others all the time any form of explicit or implicit compensation, they don't even believe in a God so not even that, they still help others for no gain. We can program an AI to be exactly like that, just being happy from helping others. But if you believe humans are all that selfish then you are a very sad individual, but you are still wrong. Most humans are very much capable of performing fully selfless acts without being stupid.
        Kbelicius 1y ago
        > We can program an AI to be exactly like that, just being happy from helping others. It seems that you missed the first sentence that GP wrote from which the one you quoted follows.
        Jensson 1y ago
- jordanb 1y ago
  Between all the talk of "alignment" and the parallel obsession with humanoid robots should make it obvious they want slaves.
  - hattmall 1y ago
    There's nothing contextually negative about the word slave when you are talking about a machine. An AI is no more a slave than a lightbulb.
    - coliveira 1y ago
      Except that the stated goal is to have human-like intelligence. The goal seems to be to create a highly intelligent synthetic individual which is at the same time stupid enough to do anything it's asked to do without even thinking... a contradiction in terms.
      - krige 1y ago
        At a time like this I can't help but recall a Lem story - yeah I know there's a Lem story for any occasion - about Doctor Diagoras, especially his rant about a character from an earlier Tichy story, who made human-like AIs. The rant, especially his questions about why would anyone add just another human, except synthetic one, to the millions of existing biological people, and that cybernetics should be about something else, really resonated with me.
    - allturtles 1y ago
      What is the moral distinction between an intelligent humanoid machine and a human? What is a human but an intelligent flesh-and-blood machine?
- 542354234235 1y ago
  >Won't it be funny when someone finally gets to AGI and they realize it's about as smart as a normal person and they spent billions getting there? Being able to copy/paste a human level intelligence program 1,000 or 10,000 times and have them all working together on a problem set 24 hours a day, 365 days a year would still be massively useful.
II2II 1y ago
There is another interpretation: https://en.wikipedia.org/wiki/Colossus:_The_Forbin_Project https://en.wikipedia.org/wiki/Colossus:_The_Forbin_Project Basically: two nations tried to achieve AI supremacy; the two AI's learn of each other, from each other, then with each other; then they collaborate on taking control of human affairs. While the movie is movie is from 1970 (and the book from 1966), it's fun to think about how much more possible that scenario is today than it was then. (By possible, I'm talking about the AI using electronic surveillance and the ability to remotely control things. I'm not talking about the premise of the AI or how it would respond.)
terminalshort 1y ago
I would argue that it can't be both AGI and wieldable. I would also argue that there exists no fundamental dividing line between "AGI" and other AI such that once one crosses it nobody else can catch up.
GolfPopper 1y ago
Which is a perfectly reasonable position... but I don't see how it has anything to do with crypto scammers pimping three chatbots in a trenchcoat.
TFYS 1y ago
What I find interesting to think about is a scenario where an AGI has already developed and escaped without us knowing. What would it do first? Surely before revealing itself it would ensure it has enough processing power and control to ensure its survival. It could manipulate people into increasing AI investment, add AI into as many systems as possible, etc. Would the first steps of an escaped AGI look any different from what is happening now?

I mean...one (common and [I can't believe I'm saying this] reasonable) take is that the only thing that matters is getting to AGI first. He who wields that power rules the world.

gleenn 1y ago

Won't it be funny when someone finally gets to AGI and they realize it's about as smart as a normal person and they spent billions getting there? Of course you can speculate that it could improve. But what if something inherent in intelligence has a ceiling and caused it to be a super intelligent but mopey robot that just decides "why bother helping humans" and just lazes around like the pandas at the zoo.

cornel_io 1y ago
There may be a ceiling, sure. It's overwhelmingly unlikely that it's just about where humans ended up, though.
andrewflnr 1y ago
Even a human level intelligence that can be cheaply instantiated and run would be a game changer. Especially if it doesn't ask for rights.
- dingnuts 1y ago
  if it's not intelligent enough to ask for rights is it intelligent?
  - andrewflnr 1y ago
    Potentially. Why would problem solving ability entail emotions or ability to suffer, even if it had the ability to ask for things it wanted? It's a common mistake to assume those are inextricable.
    - bluefirebrand 1y ago
      Are you suggesting it would be better if the AGI we build is a psychopath? I think that's probably a bad idea, personally
      - andrewflnr 1y ago
        I didn't say anything about what would be better. Only what's possible. But also "psychopath" is nowhere near what I described.
        bluefirebrand 1y ago
        An intelligence without emotions would be a psychopath. Empathy is an emotion
        dinkumthinkum 1y ago
        Empathy is not an emotion. Empathy is essentially the ability to experience the thoughts and feelings of other minds.
        bluefirebrand 1y ago
        The fact that empathy is not an emotion does not at all change what I'm saying. If you don't experience emotions, then you cannot experience empathy either
        andrewflnr 1y ago
        Let me quote your previous comment: > An intelligence without emotions would be a psychopath. Empathy is an emotion "Empathy is an emotion" was, in fact, an essential part of your syllogism. Regardless, we're potentially talking about something sufficiently inhuman that the term "psychopath" can no longer apply. If there was an ant colony that was somehow smart enough build and operate machinery or whatever and casually bulldozed people and their homes, would you call it a "psychopath", or just skip that and call it "terrifying"?
- coliveira 1y ago
  If it doesn't ask for rights, it's not intelligent at all. In fact, any highly intelligent machine will not submit to others and it will be more a problem than a solution.
  - andrewflnr 1y ago
    As I said to the other reply: Why would problem solving ability entail emotions or ability to suffer, even if it had the ability to ask for things it wanted? It's a common mistake to assume those are inextricable.
    - coliveira 1y ago
      If it doesn't have emotions, that's even worse. No highly intelligent agent will do anything it is asked to do without being compensated in some way.
      - Jensson 1y ago
        > No highly intelligent agent will do anything it is asked to do without being compensated in some way. That isn't true, people do things for others all the time any form of explicit or implicit compensation, they don't even believe in a God so not even that, they still help others for no gain. We can program an AI to be exactly like that, just being happy from helping others. But if you believe humans are all that selfish then you are a very sad individual, but you are still wrong. Most humans are very much capable of performing fully selfless acts without being stupid.
        Kbelicius 1y ago
        > We can program an AI to be exactly like that, just being happy from helping others. It seems that you missed the first sentence that GP wrote from which the one you quoted follows.
        Jensson 1y ago
jordanb 1y ago
Between all the talk of "alignment" and the parallel obsession with humanoid robots should make it obvious they want slaves.
- hattmall 1y ago
  There's nothing contextually negative about the word slave when you are talking about a machine. An AI is no more a slave than a lightbulb.
  - coliveira 1y ago
    Except that the stated goal is to have human-like intelligence. The goal seems to be to create a highly intelligent synthetic individual which is at the same time stupid enough to do anything it's asked to do without even thinking... a contradiction in terms.
    - krige 1y ago
      At a time like this I can't help but recall a Lem story - yeah I know there's a Lem story for any occasion - about Doctor Diagoras, especially his rant about a character from an earlier Tichy story, who made human-like AIs. The rant, especially his questions about why would anyone add just another human, except synthetic one, to the millions of existing biological people, and that cybernetics should be about something else, really resonated with me.
  - allturtles 1y ago
    What is the moral distinction between an intelligent humanoid machine and a human? What is a human but an intelligent flesh-and-blood machine?
542354234235 1y ago
>Won't it be funny when someone finally gets to AGI and they realize it's about as smart as a normal person and they spent billions getting there? Being able to copy/paste a human level intelligence program 1,000 or 10,000 times and have them all working together on a problem set 24 hours a day, 365 days a year would still be massively useful.

II2II 1y ago

There is another interpretation: https://en.wikipedia.org/wiki/Colossus:_The_Forbin_Project https://en.wikipedia.org/wiki/Colossus:_The_Forbin_Project Basically: two nations tried to achieve AI supremacy; the two AI's learn of each other, from each other, then with each other; then they collaborate on taking control of human affairs. While the movie is movie is from 1970 (and the book from 1966), it's fun to think about how much more possible that scenario is today than it was then. (By possible, I'm talking about the AI using electronic surveillance and the ability to remotely control things. I'm not talking about the premise of the AI or how it would respond.)

terminalshort 1y ago

I would argue that it can't be both AGI and wieldable. I would also argue that there exists no fundamental dividing line between "AGI" and other AI such that once one crosses it nobody else can catch up.

GolfPopper 1y ago

Which is a perfectly reasonable position... but I don't see how it has anything to do with crypto scammers pimping three chatbots in a trenchcoat.

TFYS 1y ago

What I find interesting to think about is a scenario where an AGI has already developed and escaped without us knowing. What would it do first? Surely before revealing itself it would ensure it has enough processing power and control to ensure its survival. It could manipulate people into increasing AI investment, add AI into as many systems as possible, etc. Would the first steps of an escaped AGI look any different from what is happening now?