4 ms·

Intelligence and Comprehension

• webmaven 4y ago
  > That is the critical element. Intelligence is connected to will, volition, the desire to do something. Where you have the “desire to do,” you also have the “desire not to do”: the ability to dissent, to disobey, to transgress. It isn’t at all surprising that the “mind control” trope is one of the most frightening in science fiction and political propaganda: that’s a direct challenge to what we see as fundamentally human. Nor is it surprising that the “disobedient computer” is another of those terrifying tropes, not because the computer can outthink us, but because by disobeying, it has become human. Personally, I find the idea of a disobedient computer far less frightening than a computer that has no agency to disregard the utility function it is slaved to. The latter is how we get a runaway paperclip maximizer: https://www.gwern.net/fiction/Clippy https://www.gwern.net/fiction/Clippy