3 ms·

Couldn't AGI itself solve the alignment problem? Why not? It's better than us by definition...

• loup-vaillant 10y ago
  How are you going to specify "solve the value alignement problem"? Even if this trick works, you have other problems, such as specifying what a human is.
  • idlewords 10y ago
    A true superintelligence should be able to figure out what people are. If it can't, we can tease it for being dumb until it becomes angry and designs a super-super-intelligence that can.
    • loup-vaillant 10y ago
      > A true superintelligence should be able to figure out what people are. Oh yes it would. How to exploit that fact into something that can reliably be carried out without misinterpretation however, is less clear. Imagine you're just telling the AI to "do what humanity wants you to". Hmm, what do we want? How individual wills can translate into collective (dis)agreement? How drives in a single individual translates into will? Depending on how the machine answers all those questions, it might perform quite differently. How about perpetuating a cycle of pain an hatred, on the basis that many people have (sometimes mortal) grudges? And even if you solve that, our current ideals may not be so ideal upon closer inspection (say, when our lives change and we take a second look). Would you want the AI freeze our current morality into an unwavering tradition? It's not clear it wouldn't. Much work to be done.