2 ms·
1. “Moral computation” was coined by Francis Hutcheson (doctoral advisor of Adam Smith). 2. I like hearing rationalists describe “demon summoning is easy, Ange
by dr_dshiv 4y ago
1. “Moral computation” was coined by Francis Hutcheson (doctoral advisor of Adam Smith).
2. I like hearing rationalists describe “demon summoning is easy, Angel summoning is hard.” Genuinely curious about that.
- xkcd-sucks 4y ago2. "An ounce of prevention is worth a pound of cure"; "measure twice, cut once"; "it's easier to fuck it than to unfuck it"; etc. Not exactly novel
- dr_dshiv 4y agoTo clarify— I mean the rationalist reference to angels and demons. It seems to be more than metaphorical.
- scarmig 4y agoAngel here would be an "aligned" AI: an AI whose behavior is in line with human values. A demon is simply an unaligned one. The key point would be that it needs not be actively hostile to human values, but indifferent to them. Think of a Lovecraftian Elder God, not AI Hitler. The supporting hypotheses would be that it's much easier to create an unaligned AI than an aligned one, as an aligned one is an unaligned one with additional constraints, and that unaligned AIs will wreak massive destruction.