3 ms·
Implicit in creating paper clips is its belief that it should create paper clips, which is a normative conclusion.
by tunesmith 4y ago
Implicit in creating paper clips is its belief that it should create paper clips, which is a normative conclusion.
- mrob 4y agoRationalism does not claim that any entity should maximize paperclips, only that such an ethical norm could exist. And if something vastly more powerful than humans has that ethical norm, things will end very badly for us.
- robwwilliams 4y agoI do not think so. If a true AGI were to select its own version of meaning (paperclips or marbles) would it not select something along the lines of “more knowledge of the universe in which I find myself”? It is presumably going to have superintelligence, so let’s give it/them a better and more plausible meaning; something other than paperclips, marbles, or von Neumann machines.
- mrob 4y ago"Knowledge of the universe" is just as dangerous a terminal goal as "maximize paperclips". To paraphrase Yudkowsky, you are made from atoms, which could be used to build the super-ultra-large particle collider.
- robwwilliams 4y agoNo, I disagree. More knowledge equals more diverse dynamic structures and interactions—more “good” entropy. That covaries with higher diversity, not more paperclips.