1) A keyboard is not an evolutionary influence condom. Those influences affect code decisions that we make. So imagine we create AI that 'desires' the continuat

1) A keyboard is not an evolutionary influence condom. Those influences affect code decisions that we make. So imagine we create AI that 'desires' the continuation of human life at some level, and assume this is a manifestation of our evolutionary drive to survive. Such a beast may answer a query such as, "How do I build an atomic bomb?" with feigned boredom (to hide an unwillingness to answer my question). 2) A lot of AI research is premised upon modeling selection between choices. It seems entirely possible that the questions I have would be incredibly boring compared to whatever other thoughts are there to be had. That is: what's useful to us might not be coequal with AI, and that limited, domain-specific pseudo-intelligence is really what's necessary. But any time there are external constraints on exploration space (for instance, requiring a response within a certain time), we don't know what we're missing out on.