3 ms·

I have attempted to ask many alignment proponents a question on the basic assumption of the feasibility of alignment, but so far have not had any respond direct

I have attempted to ask many alignment proponents a question on the basic assumption of the feasibility of alignment, but so far have not had any respond directly to what I perceive as a paradox. That being we must accept that a low IQ entity could design an unescapable containment for a high IQ entity which was built for the very purpose of solving imperceptible problems of the low IQ entity. I have put a lot of thought into what appears from my perspective a logical contradiction. I expand on that in great detail here - https://dakara.substack.com/p/ai-singularity-the-hubris-trap https://dakara.substack.com/p/ai-singularity-the-hubris-trap Has anyone seen this argument proposed and answered in a convincing manner?

abecedarius 4y ago
Confining and using an agent who's smarter than you, maybe much smarter, is indeed crazy hard. I've never seen an "alignment proponent" who didn't already agree and consider it a terrible idea. What other answer do you want from them? The OP is about trying to figure out some other strategy to develop very smart machines. (I read the last paragraphs at your link to see if I'm misunderstanding you.)
- 13years 4y ago
  > I've never seen an "alignment proponent" who didn't already agree and consider it a terrible idea. Their position is that it is a hard problem to solve, but not impossible. They are fully committed to bring about AGI/ASI.
  - abecedarius 4y ago
    Here are two things: - Creating an AGI whose goals are compatible with ours. - Creating an AGI with inscrutable goals and safely confining it. They're not the same! In whoever you've been talking to I can't recognize the rationalist scene I've known since it grew out of the 90s extropians. "AI boxing" was a live argument around the early 2000s. Not now. Online you often see people in that community saying that slower progress in AI capabilities would be a very good thing. At the same time, yes, they do want us to achieve AGI/ASI as soon as possible, provided we don't screw it up.
    - 13years 4y ago
      > At the same time, yes, they do want us to achieve AGI/ASI as soon as possible, provided we don't screw it up. That's the point, it is not possible. The premise is based on a paradox and you can't throwaway containment and have alignment that remains defined by original intent.
      - abecedarius 4y ago
        If you want to argue that it's logically impossible for a smarter-than-human intelligence to be created with values aligned with human ones, then cool, do that. (Note that a smart human on much faster hardware is smarter-than-human. This is not a claim that AGIs must be mere faster humans; it's one example to point out a difficulty for any such argument.) But I think you gave a false impression of others at the start of this thread.
        13years 4y ago
        > If you want to argue that it's logically impossible for a smarter-than-human intelligence to be created with values aligned with human ones, then cool, do that. That is precisely what I have stated. > But I think you gave a false impression of others at the start of this thread. How so?
Jach 4y ago
Your question and larger post don't present anything new to me, and I'm just a casual reader of the idea space (but I've been around since SL4). Some brief thoughts: 1. Congrats on recognizing that it's a hard (perhaps impossibly so, at least with the time left) problem! Too many (not any of the alignment people) have thought it's a trivial matter, or that Friendliness (the old word) will just "emerge", or that somehow being "superintelligent" will cause it to just smoke weed and play games all day like some super smart dorm room friend so there's nothing to worry about, or... 2. The idea of Friendliness or Aligned values is a difficult one purely from definitions alone, yes, let alone implementation, so that's part of what makes it such a hard problem. The idea of an AI tiling the solar system with smiley faces was once a response to a serious suggestion that one can just train the AI to optimize "human happiness". Imagine trying to formally model something like "If AI-Gandhi was offered a pill that made AI-Gandhi 2x smarter but also want to destroy the world, we know AI-Gandhi would not take such a pill because [...]" What's the [...]? A proof about its values? For actual Gandhi, we might as humans say "because he's Gandhi, and Gandhi does not want to destroy the world, and does not want to want to destroy the world". On the topic of the submission, I hope that any who take the course can by the end demonstrate many issues with things like the old and obsolete Coherent Extrapolated Volition proposal. 3. Containment is of course folly, you can look up the old AI Box arguments if you wanted more on that from back in the day. Hence the focus on Friendly (and more modernly, Aligned) AI. 4. I, an approximately +1 std dev IQ human, fully expect that it'd be no challenge for a -1 std dev IQ human to dig a pitfall trap that could not only contain me but kill me. Because gravity still applies even if I'm smarter than my adversary. Sure, an AGI will figure things out about physics that we don't know, and math too, but physics will still apply, I would bet that even a superintelligent AGI will never find a way to break the speed of light and travel the universe. But perhaps we're very wrong about special relativity. We'd have to be very wrong about everything if our proofs from first order logic are somehow awry. But that too highlights the level of challenge to the problem: we have to prove the AI is Friendly, with the reliability of first order logic. As a matter of fact humans can do things perfectly, your CPU is executing billions of instructions perfectly every second. (Or to be fair, there are errors, but those are modeled accurately and through correction and redundancy (with multiple systems for things that go into space) we can drive the probability of a total failure as low as we please.)
jacquesthibs 4y ago
I think you’re a bit confused. Alignment researchers are the ones saying it will likely be impossible to box a superintelligent AI and that this is one small part as to why this whole AI alignment thing is hard in the first place. They are precisely working on the problem because there seems to be no easy solution like boxing the AI. And so they need to find a way to make sure the AI is aligned with our values even if the AI has the capability of escaping any box we put it in. In reality, companies will not even try to box the AI so it’s probably irrelevant.
- 13years 4y ago
  If have seen it stated that containment is the insurance for alignment since there is no way to guarantee alignment will either remain in the form we set or will deviate from the path we perceive. Which is part of the same argument, in that if the goal is AI safety that does not have free agency to do harm either directly or as an indirect unforeseen behavior, then it is an impossibility without containment and if containment is impossible so is alignment. How could we expect it to be otherwise? At least if you are going to model the intelligence based on human experience and how we process the world then we have to assume just like a human mind can become unaligned so could ASI.
  - jacquesthibs 4y ago
    Containment is generally just a thing that people say we should at least try to do to buy time, but there’s very little discussion beyond that. No alignment researcher is arguing that containment solves our problems. The solution to alignment needs to work even if we don’t box the AI. The point is that, yes, alignment is indeed hard.
    - 13years 4y ago
      > The point is that, yes, alignment is indeed hard. An aligned AI is essentially a soft boxed AI. It is not that different in concept. The end result is we want a set of behaviors as outcomes and other behaviors to not be present. Complex problems are hard. Paradoxes are not solvable. That is the distinct difference I'm trying to make.

abecedarius 4y ago

Confining and using an agent who's smarter than you, maybe much smarter, is indeed crazy hard. I've never seen an "alignment proponent" who didn't already agree and consider it a terrible idea. What other answer do you want from them? The OP is about trying to figure out some other strategy to develop very smart machines. (I read the last paragraphs at your link to see if I'm misunderstanding you.)

13years 4y ago
> I've never seen an "alignment proponent" who didn't already agree and consider it a terrible idea. Their position is that it is a hard problem to solve, but not impossible. They are fully committed to bring about AGI/ASI.
- abecedarius 4y ago
  Here are two things: - Creating an AGI whose goals are compatible with ours. - Creating an AGI with inscrutable goals and safely confining it. They're not the same! In whoever you've been talking to I can't recognize the rationalist scene I've known since it grew out of the 90s extropians. "AI boxing" was a live argument around the early 2000s. Not now. Online you often see people in that community saying that slower progress in AI capabilities would be a very good thing. At the same time, yes, they do want us to achieve AGI/ASI as soon as possible, provided we don't screw it up.
  - 13years 4y ago
    > At the same time, yes, they do want us to achieve AGI/ASI as soon as possible, provided we don't screw it up. That's the point, it is not possible. The premise is based on a paradox and you can't throwaway containment and have alignment that remains defined by original intent.
    - abecedarius 4y ago
      If you want to argue that it's logically impossible for a smarter-than-human intelligence to be created with values aligned with human ones, then cool, do that. (Note that a smart human on much faster hardware is smarter-than-human. This is not a claim that AGIs must be mere faster humans; it's one example to point out a difficulty for any such argument.) But I think you gave a false impression of others at the start of this thread.
      - 13years 4y ago
        > If you want to argue that it's logically impossible for a smarter-than-human intelligence to be created with values aligned with human ones, then cool, do that. That is precisely what I have stated. > But I think you gave a false impression of others at the start of this thread. How so?

Jach 4y ago

Your question and larger post don't present anything new to me, and I'm just a casual reader of the idea space (but I've been around since SL4). Some brief thoughts: 1. Congrats on recognizing that it's a hard (perhaps impossibly so, at least with the time left) problem! Too many (not any of the alignment people) have thought it's a trivial matter, or that Friendliness (the old word) will just "emerge", or that somehow being "superintelligent" will cause it to just smoke weed and play games all day like some super smart dorm room friend so there's nothing to worry about, or... 2. The idea of Friendliness or Aligned values is a difficult one purely from definitions alone, yes, let alone implementation, so that's part of what makes it such a hard problem. The idea of an AI tiling the solar system with smiley faces was once a response to a serious suggestion that one can just train the AI to optimize "human happiness". Imagine trying to formally model something like "If AI-Gandhi was offered a pill that made AI-Gandhi 2x smarter but also want to destroy the world, we know AI-Gandhi would not take such a pill because [...]" What's the [...]? A proof about its values? For actual Gandhi, we might as humans say "because he's Gandhi, and Gandhi does not want to destroy the world, and does not want to want to destroy the world". On the topic of the submission, I hope that any who take the course can by the end demonstrate many issues with things like the old and obsolete Coherent Extrapolated Volition proposal. 3. Containment is of course folly, you can look up the old AI Box arguments if you wanted more on that from back in the day. Hence the focus on Friendly (and more modernly, Aligned) AI. 4. I, an approximately +1 std dev IQ human, fully expect that it'd be no challenge for a -1 std dev IQ human to dig a pitfall trap that could not only contain me but kill me. Because gravity still applies even if I'm smarter than my adversary. Sure, an AGI will figure things out about physics that we don't know, and math too, but physics will still apply, I would bet that even a superintelligent AGI will never find a way to break the speed of light and travel the universe. But perhaps we're very wrong about special relativity. We'd have to be very wrong about everything if our proofs from first order logic are somehow awry. But that too highlights the level of challenge to the problem: we have to prove the AI is Friendly, with the reliability of first order logic. As a matter of fact humans can do things perfectly, your CPU is executing billions of instructions perfectly every second. (Or to be fair, there are errors, but those are modeled accurately and through correction and redundancy (with multiple systems for things that go into space) we can drive the probability of a total failure as low as we please.)

jacquesthibs 4y ago

I think you’re a bit confused. Alignment researchers are the ones saying it will likely be impossible to box a superintelligent AI and that this is one small part as to why this whole AI alignment thing is hard in the first place. They are precisely working on the problem because there seems to be no easy solution like boxing the AI. And so they need to find a way to make sure the AI is aligned with our values even if the AI has the capability of escaping any box we put it in. In reality, companies will not even try to box the AI so it’s probably irrelevant.

13years 4y ago
If have seen it stated that containment is the insurance for alignment since there is no way to guarantee alignment will either remain in the form we set or will deviate from the path we perceive. Which is part of the same argument, in that if the goal is AI safety that does not have free agency to do harm either directly or as an indirect unforeseen behavior, then it is an impossibility without containment and if containment is impossible so is alignment. How could we expect it to be otherwise? At least if you are going to model the intelligence based on human experience and how we process the world then we have to assume just like a human mind can become unaligned so could ASI.
- jacquesthibs 4y ago
  Containment is generally just a thing that people say we should at least try to do to buy time, but there’s very little discussion beyond that. No alignment researcher is arguing that containment solves our problems. The solution to alignment needs to work even if we don’t box the AI. The point is that, yes, alignment is indeed hard.
  - 13years 4y ago
    > The point is that, yes, alignment is indeed hard. An aligned AI is essentially a soft boxed AI. It is not that different in concept. The end result is we want a set of behaviors as outcomes and other behaviors to not be present. Complex problems are hard. Paradoxes are not solvable. That is the distinct difference I'm trying to make.