3 ms·

Confining and using an agent who's smarter than you, maybe much smarter, is indeed crazy hard. I've never seen an "alignment proponent" who didn't already agree

Confining and using an agent who's smarter than you, maybe much smarter, is indeed crazy hard. I've never seen an "alignment proponent" who didn't already agree and consider it a terrible idea. What other answer do you want from them? The OP is about trying to figure out some other strategy to develop very smart machines. (I read the last paragraphs at your link to see if I'm misunderstanding you.)

13years 4y ago
> I've never seen an "alignment proponent" who didn't already agree and consider it a terrible idea. Their position is that it is a hard problem to solve, but not impossible. They are fully committed to bring about AGI/ASI.
- abecedarius 4y ago
  Here are two things: - Creating an AGI whose goals are compatible with ours. - Creating an AGI with inscrutable goals and safely confining it. They're not the same! In whoever you've been talking to I can't recognize the rationalist scene I've known since it grew out of the 90s extropians. "AI boxing" was a live argument around the early 2000s. Not now. Online you often see people in that community saying that slower progress in AI capabilities would be a very good thing. At the same time, yes, they do want us to achieve AGI/ASI as soon as possible, provided we don't screw it up.
  - 13years 4y ago
    > At the same time, yes, they do want us to achieve AGI/ASI as soon as possible, provided we don't screw it up. That's the point, it is not possible. The premise is based on a paradox and you can't throwaway containment and have alignment that remains defined by original intent.
    - abecedarius 4y ago
      If you want to argue that it's logically impossible for a smarter-than-human intelligence to be created with values aligned with human ones, then cool, do that. (Note that a smart human on much faster hardware is smarter-than-human. This is not a claim that AGIs must be mere faster humans; it's one example to point out a difficulty for any such argument.) But I think you gave a false impression of others at the start of this thread.
      - 13years 4y ago
        > If you want to argue that it's logically impossible for a smarter-than-human intelligence to be created with values aligned with human ones, then cool, do that. That is precisely what I have stated. > But I think you gave a false impression of others at the start of this thread. How so?

13years 4y ago

> I've never seen an "alignment proponent" who didn't already agree and consider it a terrible idea. Their position is that it is a hard problem to solve, but not impossible. They are fully committed to bring about AGI/ASI.

abecedarius 4y ago
Here are two things: - Creating an AGI whose goals are compatible with ours. - Creating an AGI with inscrutable goals and safely confining it. They're not the same! In whoever you've been talking to I can't recognize the rationalist scene I've known since it grew out of the 90s extropians. "AI boxing" was a live argument around the early 2000s. Not now. Online you often see people in that community saying that slower progress in AI capabilities would be a very good thing. At the same time, yes, they do want us to achieve AGI/ASI as soon as possible, provided we don't screw it up.
- 13years 4y ago
  > At the same time, yes, they do want us to achieve AGI/ASI as soon as possible, provided we don't screw it up. That's the point, it is not possible. The premise is based on a paradox and you can't throwaway containment and have alignment that remains defined by original intent.
  - abecedarius 4y ago
    If you want to argue that it's logically impossible for a smarter-than-human intelligence to be created with values aligned with human ones, then cool, do that. (Note that a smart human on much faster hardware is smarter-than-human. This is not a claim that AGIs must be mere faster humans; it's one example to point out a difficulty for any such argument.) But I think you gave a false impression of others at the start of this thread.
    - 13years 4y ago
      > If you want to argue that it's logically impossible for a smarter-than-human intelligence to be created with values aligned with human ones, then cool, do that. That is precisely what I have stated. > But I think you gave a false impression of others at the start of this thread. How so?