3 ms·
Because computer programs do what we said, not what we meant. If you accidentally write an infinite loop somewhere, your program will get stuck there. If you as
by mitthrowaway2 3y ago
Because computer programs do what we said, not what we meant. If you accidentally write an infinite loop somewhere, your program will get stuck there. If you ask for "some" paperclips, you'll possibly be fine; on the other hand if you carelessly asked it to maximize something, such as paperclips-per-month or paperclips-per-dollar, then it might go far beyond where human common sense would stop.
- Tokumei-no-hito 3y ago> do whatever is necessary to guarantee consistently increasing quarterly profits Seems like a likely request
- goatlover 3y agoWe're talking about a Super AI, not normal computer programs. Ask ChatGPT to maximize paperclip making and see what it says.
- mitthrowaway2 3y agoChatGPT is not a general intelligence, let alone a superintelligence. Ask for paperclips and it will try to provide a plausible, satisfying, and entertaining answer. To put it another way, my parents wanted me to become a Jewish doctor but I became an atheist engineer. Was I simply not smart enough to understand what they wanted?