TL;DR: "Obeying orders" is a powerful pattern; combined with "computer is always right", it can be downright deadly. An anecdote: I've been driving in this one

TL;DR: "Obeying orders" is a powerful pattern; combined with "computer is always right", it can be downright deadly. An anecdote: I've been driving in this one particular city for ages, I know the streets and their properties - yet while I was busy with traffic (rain and night and pedestrians, oh my), it only took a "turn left NOW" command to send me wrong way into a one-way street (instead of the correct one, a block down). I knew that specific street was one way, but not focusing on navigation at that moment, I have blindly obeyed, as it looked somewhat reasonable; only after turning did I realize what I have done (fortunately, nothing else came of that incident). Note that I do not have blind trust in algorithms - quite the contrary (turned out the navigation's data was wrong), yet I have outsourced a part of my cognitive load onto the device. I'm not blaming the device: the faulty action was all mine (that's what you get for a talking toaster ordering you around). What I got was some insight: I no longer wonder why someone with more trust in their computing devices could do something so "blindingly stupid." This is probably the one area in UX that's hard to fix: the assumption that Friend Computer Is Always Right.