There is a distinction between "terminal" and "instrumental" values. If you continually ask yourself questions like "why am I doing this", you will eventually g

There is a distinction between "terminal" and "instrumental" values. If you continually ask yourself questions like "why am I doing this", you will eventually get an answer of "just because". That is your terminal value, the reason you do everything you do. Humans have some complicated unknown set of terminal values. Everything we do, we do because we believe it will maximize our values. A value might be something like "avoid suffering", or "other people suffering is bad", etc. In this example the AIs terminal value is "make paperclips". Everything it does, it does because it believes it will maximize this value. It would never ask itself questions like "why", because that action doesn't maximize that value. If it has the ability to rewrite it's own code, it would rewrite it in such a way that it can't ever change it's own goals. Changing goals is counterproductive to achieving them.