There's another aspect of the "scary sentient AI" trope that I think about a lot. It's not about the danger, but the surprise. People don't expect to be surpris

There's another aspect of the "scary sentient AI" trope that I think about a lot. It's not about the danger, but the surprise. People don't expect to be surprised by things that aren't other animals, or the weather. It doesn't matter whether AI is "conscious" or "intelligent" if it seems intelligent. Imagine how surprising an automatic door opener would be to a person living 150 years ago. It would seem as if the door "knew" they were approaching, and politely opened for them. Imagine how surprised we would be if Clippy understood what you were doing, and offered help that was actually helpful. As AI advances and is incorporated more and more into everyday things, our environment will become a reactive, "intelligent" black box, and along the way there will be some sudden advances that may be very startling and surreal.