2 ms·

> I've never seen an "alignment proponent" who didn't already agree and consider it a terrible idea. Their position is that it is a hard problem to solve, but

> I've never seen an "alignment proponent" who didn't already agree and consider it a terrible idea. Their position is that it is a hard problem to solve, but not impossible. They are fully committed to bring about AGI/ASI.

abecedarius 4y ago
Here are two things: - Creating an AGI whose goals are compatible with ours. - Creating an AGI with inscrutable goals and safely confining it. They're not the same! In whoever you've been talking to I can't recognize the rationalist scene I've known since it grew out of the 90s extropians. "AI boxing" was a live argument around the early 2000s. Not now. Online you often see people in that community saying that slower progress in AI capabilities would be a very good thing. At the same time, yes, they do want us to achieve AGI/ASI as soon as possible, provided we don't screw it up.
- 13years 4y ago
  > At the same time, yes, they do want us to achieve AGI/ASI as soon as possible, provided we don't screw it up. That's the point, it is not possible. The premise is based on a paradox and you can't throwaway containment and have alignment that remains defined by original intent.
  - abecedarius 4y ago
    If you want to argue that it's logically impossible for a smarter-than-human intelligence to be created with values aligned with human ones, then cool, do that. (Note that a smart human on much faster hardware is smarter-than-human. This is not a claim that AGIs must be mere faster humans; it's one example to point out a difficulty for any such argument.) But I think you gave a false impression of others at the start of this thread.
    - 13years 4y ago
      > If you want to argue that it's logically impossible for a smarter-than-human intelligence to be created with values aligned with human ones, then cool, do that. That is precisely what I have stated. > But I think you gave a false impression of others at the start of this thread. How so?

abecedarius 4y ago

Here are two things: - Creating an AGI whose goals are compatible with ours. - Creating an AGI with inscrutable goals and safely confining it. They're not the same! In whoever you've been talking to I can't recognize the rationalist scene I've known since it grew out of the 90s extropians. "AI boxing" was a live argument around the early 2000s. Not now. Online you often see people in that community saying that slower progress in AI capabilities would be a very good thing. At the same time, yes, they do want us to achieve AGI/ASI as soon as possible, provided we don't screw it up.

13years 4y ago
> At the same time, yes, they do want us to achieve AGI/ASI as soon as possible, provided we don't screw it up. That's the point, it is not possible. The premise is based on a paradox and you can't throwaway containment and have alignment that remains defined by original intent.
- abecedarius 4y ago
  If you want to argue that it's logically impossible for a smarter-than-human intelligence to be created with values aligned with human ones, then cool, do that. (Note that a smart human on much faster hardware is smarter-than-human. This is not a claim that AGIs must be mere faster humans; it's one example to point out a difficulty for any such argument.) But I think you gave a false impression of others at the start of this thread.
  - 13years 4y ago
    > If you want to argue that it's logically impossible for a smarter-than-human intelligence to be created with values aligned with human ones, then cool, do that. That is precisely what I have stated. > But I think you gave a false impression of others at the start of this thread. How so?