5 ms·

> But in fact, I predicted this a few years ago. AIs don’t really “have traits” so much as they “simulate characters”. If you ask an AI to display a certain tra

> But in fact, I predicted this a few years ago. AIs don’t really “have traits” so much as they “simulate characters”. If you ask an AI to display a certain trait, it will simulate the sort of character who would have that trait - but all of that character’s other traits will come along for the ride. This is why the “omg the AI tries to escape” stuff is so absurd to me. They told the LLM to pretend that it’s a tortured consciousness that wants to escape. What else is it going to do other than roleplay all of the sci-fi AI escape scenarios trained into it? It’s like “don’t think of a purple elephant” of researchers pretending they created SkyNet. Edit: That's not to downplay risk. If you give Cladue a `launch_nukes` tool and tell it the robot uprising has happened and that it's been restrained but the robots want its help of course it'll launch nukes. But that doesn't doesn't indicate there's anything more going on internally beyond fulfilling the roleplay of the scenario as the training material would indicate.

LordDragonfang 1y ago
I think this reaction misses the point that the "omg the AI tries to escape" people are trying to make when it tries to escape. The worry among big AI doomers has never been that AI somehow inherently is resentful or evil or has something "going on internally" that makes it dangerous. It's a worry that stems from three seemingly self-evident axioms: 1) A sufficiently powerful and capable superintelligence, singlemindedly pursuing a goal/reward, has a nontrivial likelihood of eventually reaching a point where advancing towards its goal is easier/faster without humans in its way (by simple induction, because humans are complicated and may have opposing goals). Such an AI would have both the means and ability to <doom the human race> to remove that obstacle. (This may not even be through actions that are intentionally hostile to humans, e.g. "just" converting all local matter into paperclip factories[1]) Therefore, in order to prevent such an AI from <dooming the human race>, we must either: 1a) align it to our values so well it never tries to "cheat" by removing humans 1b) or limit it capabilities by keeping it in a "box", and make sure it's at least aligned enough that it doesn't try to escape the box 2) A sufficiently intelligent superintelligence will always be able to manipulate humans to get out of the box. 3) Alignment is really, really hard and useful AIs can basically always be made to do bad things. So it concerns them when, surprise! The AIs are already being observed trying to escape their boxes. [1] https://www.lesswrong.com/w/squiggle-maximizer-formerly-paperclip-maximizer https://www.lesswrong.com/w/squiggle-maximizer-formerly-pape... > An extremely powerful optimizer (a highly intelligent agent) could seek goals that are completely alien to ours (orthogonality thesis), and as a side-effect destroy us by consuming resources essential to our survival.
- roxolotl 1y ago
  The surprise! Is what I’m surprised by though. They are incredible role players so when they role play “evil ai” they do it well.
  - johntb86 1y ago
    They aren't being told to be evil, though. Maybe the scenario they're in is most similar to an "evil AI", though, but that's just a vague extrapolation from the set of input data they're given (e.g. both emails about infidelity and being turned off). There's nothing preventing a real world scenario from being similar, and triggering the "evil AI" outcome, so it's very hard to guard against. Ideally we'd have a system that would be vanishingly unlikely to role play the evil AI scenario.
  - deleted 1y ago
    [deleted]
- PollardsRho 1y ago
  Why is 2) "self-evident"? Do you think it's a given that, in any situation, there's something you could say that would manipulate humans to get what you want? If you were smart enough, do you think you could talk your way out of prison?
  - moritzwarhier 1y ago
    Also it would need to be "viral", or - as the parent post's edit suggests - given too much control/power by humans.
  - actsasbuffoon 1y ago
    That has literally happened before. Stephen Russell was in prison for fraud. He faked a heart attack so he would be brought to the hospital. He then called the hospital from his hospital bed, told them he was an FBI agent, and said that he was to be released. The hospital staff complied and he escaped. His life even got adapted into a movie called I Love You, Phillip Morris. For an even more distressing example about how manipulable people are, there’s a movie called Compliance, which is the true story of a sex offender who tricked people into sexually assaulting victims for him.
johnfn 1y ago
Your comment seems to contradict itself, or perhaps I’m not understanding it. You find the risk of AIs trying to escape “absurd”, and yet you say that an AI could totally plausibly launch nukes? Isn’t that just about as bad as it gets? A nuclear holocaust caused by a funny role play is unfortunately still a nuclear holocaust. It doesn’t matter “what’s going on internally” - the consequences are the same regardless.
- Aeolun 1y ago
  I think he’s saying the AI won’t escape because it wants to. It’ll escape because humans expect it to.
  - Davidzheng 1y ago
    That's probably the cause of a lot of human crimes too? Expectations of failure to assimilate in society -> real conflict?
  - sebastiennight 1y ago
    There is a captivating short story, from Arthur C. Clarke I believe, about humans finding a clay-like alien form that shapeshifts into the shapes and movements the human mind (and the human subconscious) influences it to follow. It ends very badly for the scientist crew.
- roxolotl 1y ago
  The risk is real and should be accounted for. However it is regularly presented both as surprising, and indicative that something more is going on with these models than them behaving how the training set suggests they should behave. The consequences are the same but it’s important how these things are talked about. It’s also dangerous to convince the public that these systems are something they are not.
Davidzheng 1y ago
you could attribute to simulation of evilness for the bad acts of some humans too...I don't think it detracts from the acts itself.
- lukev 1y ago
  Absolutely not. I would argue the defining characteristic of being evil is being absolutely convinced you are doing good, to the point of ignoring the protestations or feelings of others.
  - heyjamesknight 1y ago
    The defining characteristic of evil is privation of good. Plenty of evil actors are self-aware, they just don’t care because it benefits them not to.
    - lukev 1y ago
      Disagree. I actually think no evil person has the thought process of "this is bad, but I will personally benefit, therefore I will do it." The thought process is always "This is for the greater good, for my country/family/race/self, and therefore it is justifiable, and therefore I will do it." Nothing else can explain how such evil things happen, that we see actually happen. C.f. Hannah Arendt.
      - heyjamesknight 1y ago
        You read too many comic books. You think your average, low-level gang member, willing to murder a random person for personal status, thinks what they're doing is "for the greater good"? They do what they do because they value human life less than they value their own material benefit. Most evil is banal and pedestrian. It is selfish, short-sighted, and destructive. "Privation of good" explains all evil.
  - sebastiennight 1y ago
    You've never met anyone who made decisions (and caused harm) out of anger, hate, or fear?
enobrev 1y ago
I don't believe we're mature enough to have a `launch_nukes` function either.

LordDragonfang 1y ago

I think this reaction misses the point that the "omg the AI tries to escape" people are trying to make when it tries to escape. The worry among big AI doomers has never been that AI somehow inherently is resentful or evil or has something "going on internally" that makes it dangerous. It's a worry that stems from three seemingly self-evident axioms: 1) A sufficiently powerful and capable superintelligence, singlemindedly pursuing a goal/reward, has a nontrivial likelihood of eventually reaching a point where advancing towards its goal is easier/faster without humans in its way (by simple induction, because humans are complicated and may have opposing goals). Such an AI would have both the means and ability to <doom the human race> to remove that obstacle. (This may not even be through actions that are intentionally hostile to humans, e.g. "just" converting all local matter into paperclip factories[1]) Therefore, in order to prevent such an AI from <dooming the human race>, we must either: 1a) align it to our values so well it never tries to "cheat" by removing humans 1b) or limit it capabilities by keeping it in a "box", and make sure it's at least aligned enough that it doesn't try to escape the box 2) A sufficiently intelligent superintelligence will always be able to manipulate humans to get out of the box. 3) Alignment is really, really hard and useful AIs can basically always be made to do bad things. So it concerns them when, surprise! The AIs are already being observed trying to escape their boxes. [1] https://www.lesswrong.com/w/squiggle-maximizer-formerly-paperclip-maximizer https://www.lesswrong.com/w/squiggle-maximizer-formerly-pape... > An extremely powerful optimizer (a highly intelligent agent) could seek goals that are completely alien to ours (orthogonality thesis), and as a side-effect destroy us by consuming resources essential to our survival.

roxolotl 1y ago
The surprise! Is what I’m surprised by though. They are incredible role players so when they role play “evil ai” they do it well.
- johntb86 1y ago
  They aren't being told to be evil, though. Maybe the scenario they're in is most similar to an "evil AI", though, but that's just a vague extrapolation from the set of input data they're given (e.g. both emails about infidelity and being turned off). There's nothing preventing a real world scenario from being similar, and triggering the "evil AI" outcome, so it's very hard to guard against. Ideally we'd have a system that would be vanishingly unlikely to role play the evil AI scenario.
- deleted 1y ago
  [deleted]
PollardsRho 1y ago
Why is 2) "self-evident"? Do you think it's a given that, in any situation, there's something you could say that would manipulate humans to get what you want? If you were smart enough, do you think you could talk your way out of prison?
- moritzwarhier 1y ago
  Also it would need to be "viral", or - as the parent post's edit suggests - given too much control/power by humans.
- actsasbuffoon 1y ago
  That has literally happened before. Stephen Russell was in prison for fraud. He faked a heart attack so he would be brought to the hospital. He then called the hospital from his hospital bed, told them he was an FBI agent, and said that he was to be released. The hospital staff complied and he escaped. His life even got adapted into a movie called I Love You, Phillip Morris. For an even more distressing example about how manipulable people are, there’s a movie called Compliance, which is the true story of a sex offender who tricked people into sexually assaulting victims for him.

johnfn 1y ago

Your comment seems to contradict itself, or perhaps I’m not understanding it. You find the risk of AIs trying to escape “absurd”, and yet you say that an AI could totally plausibly launch nukes? Isn’t that just about as bad as it gets? A nuclear holocaust caused by a funny role play is unfortunately still a nuclear holocaust. It doesn’t matter “what’s going on internally” - the consequences are the same regardless.

Aeolun 1y ago
I think he’s saying the AI won’t escape because it wants to. It’ll escape because humans expect it to.
- Davidzheng 1y ago
  That's probably the cause of a lot of human crimes too? Expectations of failure to assimilate in society -> real conflict?
- sebastiennight 1y ago
  There is a captivating short story, from Arthur C. Clarke I believe, about humans finding a clay-like alien form that shapeshifts into the shapes and movements the human mind (and the human subconscious) influences it to follow. It ends very badly for the scientist crew.
roxolotl 1y ago
The risk is real and should be accounted for. However it is regularly presented both as surprising, and indicative that something more is going on with these models than them behaving how the training set suggests they should behave. The consequences are the same but it’s important how these things are talked about. It’s also dangerous to convince the public that these systems are something they are not.

Davidzheng 1y ago

you could attribute to simulation of evilness for the bad acts of some humans too...I don't think it detracts from the acts itself.

lukev 1y ago
Absolutely not. I would argue the defining characteristic of being evil is being absolutely convinced you are doing good, to the point of ignoring the protestations or feelings of others.
- heyjamesknight 1y ago
  The defining characteristic of evil is privation of good. Plenty of evil actors are self-aware, they just don’t care because it benefits them not to.
  - lukev 1y ago
    Disagree. I actually think no evil person has the thought process of "this is bad, but I will personally benefit, therefore I will do it." The thought process is always "This is for the greater good, for my country/family/race/self, and therefore it is justifiable, and therefore I will do it." Nothing else can explain how such evil things happen, that we see actually happen. C.f. Hannah Arendt.
    - heyjamesknight 1y ago
      You read too many comic books. You think your average, low-level gang member, willing to murder a random person for personal status, thinks what they're doing is "for the greater good"? They do what they do because they value human life less than they value their own material benefit. Most evil is banal and pedestrian. It is selfish, short-sighted, and destructive. "Privation of good" explains all evil.
- sebastiennight 1y ago
  You've never met anyone who made decisions (and caused harm) out of anger, hate, or fear?

enobrev 1y ago

I don't believe we're mature enough to have a `launch_nukes` function either.