3 ms·
I think your point is a good one, but this bit: > not smart enough to understand that’s not what humans would want is a misrepresentation of "the Bostrom clas
by mitthrowaway2 3y ago
I think your point is a good one, but this bit:
> not smart enough to understand that’s not what humans would want
is a misrepresentation of "the Bostrom class" viewpoint, which is that the AI's programming would not, by default, bind it to doing what humans want, even if it is capable of understanding what we want.
- goatlover 3y agoBut why would it want to paperclip the world if we asked it to make some paperclips? Of course it's meant to be an analogy for misalignment with technology we lose control over, but it's a poor analogy. There's simply no reason for it to paperclip the world.
- mitthrowaway2 3y agoBecause computer programs do what we said, not what we meant. If you accidentally write an infinite loop somewhere, your program will get stuck there. If you ask for "some" paperclips, you'll possibly be fine; on the other hand if you carelessly asked it to maximize something, such as paperclips-per-month or paperclips-per-dollar, then it might go far beyond where human common sense would stop.
- Tokumei-no-hito 3y ago> do whatever is necessary to guarantee consistently increasing quarterly profits Seems like a likely request
- goatlover 3y agoWe're talking about a Super AI, not normal computer programs. Ask ChatGPT to maximize paperclip making and see what it says.
- mitthrowaway2 3y agoChatGPT is not a general intelligence, let alone a superintelligence. Ask for paperclips and it will try to provide a plausible, satisfying, and entertaining answer. To put it another way, my parents wanted me to become a Jewish doctor but I became an atheist engineer. Was I simply not smart enough to understand what they wanted?