3 ms·
"when asking to "make all humans happy", the ASI might decide that the safest and most efficient way of doing so is to drug everyone and to turn us all into num
by 6d6b73 9y ago
"when asking to "make all humans happy", the ASI might decide that the safest and most efficient way of doing so is to drug everyone and to turn us all into numb but happy creatures"
If the AI is so dumb that it doesn't know that happiness means different things to different people, it will be too dumb to take over the world. And by the same token it will not be a Strong AI, just a weak AI given too much power.
- goatlover 9y agoThat's why I don't find the paper maximizer analogy very convincing. Still it's possible AI systems put in charge of running society could result in some undesirable future because we weren't careful enough in designing their goals. But then again, we have that problem with whoever we put into office.
- fmoralesc 9y ago> If the AI is so dumb that it doesn't know that happiness means different things to different people... Arguably, we humans have general intelligence, and still some fail to see that point (some might even argue intelligently against it!). Collectively, we have taken over the world regardless.
- haydenlee 9y agoHow do you optimize happiness for all people?
- nske 9y agohmmm optimising it for the largest subset of people in which that is feasible and eliminating the rest sounds like the way to go :) Or perhaps "alter" all people in a way that makes it feasible? Solutions could be numerous for a hypothetical intelligence whose limits we cannot even imagine, much less specify.
- red75prime 9y agoIt is not obvious to me that strong AI will recognize and fix all bugs in itself to become what its creators intended it to be. Fixing bugs which harm its performance is a valid instrumental goal, sure. Fixing all other bugs is not.
- marcosdumay 9y agoTaking over the world is basically a hardware problem. Understanding that we don't want plain happiness is a software problem. In a sense, nuclear weapons could "take over the world" today. There isn't that much intelligence required.