To be more specific, it's an argument against the idea of Bostrom/Yudkowsky-style recursively self-improving superintelligence, i.e. the brain surgeon who repea

To be more specific, it's an argument against the idea of Bostrom/Yudkowsky-style recursively self-improving superintelligence, i.e. the brain surgeon who repeatedly operates on the part of his own brain that makes him good at brain surgery, getting faster each time. It's also a cry of impatience against people who think they can model or forecast the actions of a non-human intelligence, let alone a superintelligence. AIs are alien sociopaths; it's a category error to believe you can get inside their head.