One of the biggest problems with thought experiments designed as an introduction to an entire class of problems, is that they get misinterpreted as a complete s

One of the biggest problems with thought experiments designed as an introduction to an entire class of problems, is that they get misinterpreted as a complete statement of the problem. Charlie Stross is quite dismissive of paperclip maximisers: Elon Musk "has an obsessive fear" of them, and "isn't paying enough attention" because Tesla does the same thing. He refers to a "pure paperclip maximiser" and discusses the "naive vision of a paperclip maximiser". This is quite insulting, because the paperclip maximiser is a thought experiment designed to introduce the consequences of intelligences which have fundamentally different values to human beings, in an accessible way. What he's doing is like reading a kids' book on counting and then writing an article contrasting naive apple-based addition with his brilliant new idea of generic fruit-based counting.