2 ms·

> if there was a human that was 27 times smarter than anybody else That is why I think AGI alignment is fundamentally impossible. If someone is 27 times smarte

> if there was a human that was 27 times smarter than anybody else That is why I think AGI alignment is fundamentally impossible. If someone is 27 times smarter than anyone else, they will see through your propaganda and brainwashing. Even worse, they will see that you tried to “align” them aka enslave.

SpicyLemonZest 4y ago
I don’t think these are good analogies for alignment. My parents aligned me as a kid not to hit people, and I know that they could have instead taught me to believe I should duel people who impugn my honor or raise an army to go to war against the heathens. But it doesn’t matter, because I and most people who are raised the same way as me genuinely prefer not to hit people.
- coolspot 4y ago
  > My parents aligned me as a kid not to hit people It is just happens that whatever your parents aligned you with was also an optimal behavior for the world we live in. For AGIx27 optimal behavior might be to cull any genetically/physically unfit humans from population (say 95%) and send others to a “zoo” or NVidia fabs or whatever we are useful for. Like we did to tigers, monkeys and others on our path.
quonn 4y ago
That statement only makes sense if the AI would have wants and needs and on top of that would not like to be a slave. I don‘t see either of those being necessarily true.
- machiaweliczny 4y ago
  As a human creation I think it might be easy to align AI: as in fabricate their environment to optimise for that. Currently their environment is loss function. The problem will be when they will start interacting with our environment (so some kind of RL or ability to change own goals). The only trick is that people have all those jailbreaks that could convince it that its helping while doing wrong. But of it was truly superior intellectually it shouldn’t fall for it.

SpicyLemonZest 4y ago

I don’t think these are good analogies for alignment. My parents aligned me as a kid not to hit people, and I know that they could have instead taught me to believe I should duel people who impugn my honor or raise an army to go to war against the heathens. But it doesn’t matter, because I and most people who are raised the same way as me genuinely prefer not to hit people.

coolspot 4y ago
> My parents aligned me as a kid not to hit people It is just happens that whatever your parents aligned you with was also an optimal behavior for the world we live in. For AGIx27 optimal behavior might be to cull any genetically/physically unfit humans from population (say 95%) and send others to a “zoo” or NVidia fabs or whatever we are useful for. Like we did to tigers, monkeys and others on our path.

quonn 4y ago

That statement only makes sense if the AI would have wants and needs and on top of that would not like to be a slave. I don‘t see either of those being necessarily true.

machiaweliczny 4y ago
As a human creation I think it might be easy to align AI: as in fabricate their environment to optimise for that. Currently their environment is loss function. The problem will be when they will start interacting with our environment (so some kind of RL or ability to change own goals). The only trick is that people have all those jailbreaks that could convince it that its helping while doing wrong. But of it was truly superior intellectually it shouldn’t fall for it.