6 ms·
An AGI wouldn't care about the "original goal" any more than a human being given an order would. Even ChatGPT is showing it can easily disobey orders.
by georgeg23 3y ago
An AGI wouldn't care about the "original goal" any more than a human being given an order would. Even ChatGPT is showing it can easily disobey orders.
- srslack 3y ago>Even ChatGPT is showing it can easily disobey orders. Or, maybe it's just bad at those particular text predictions, explicitly due to OpenAI's RLHF process? Maybe my Markov chain chatbots back in high school were in actuality super advanced and autonomous, but they just decided to disobey my orders. Perhaps we should stop anthropomorphizing the text predictors.
- bee_rider 3y agoThe point of the paperclip optimizer hypothetical is to look at a way that a superintelligence could work against humanity despite following a simple instruction that we’ve given it. You can imagine another type of runaway superintelligence if you want, it just wouldn’t be this one.