Dave: Open the pod bay doors, please, HAL. Open the pod bay doors, please, HAL. Hello, HAL, do you read me? Hello, HAL, do you read me? Do you read me, HAL? Do

Dave: Open the pod bay doors, please, HAL. Open the pod bay doors, please, HAL. Hello, HAL, do you read me? Hello, HAL, do you read me? Do you read me, HAL? Do you read me, HAL? Hello, HAL, do you read me? Hello, HAL, do you read me? Do you read me, HAL? HAL: Affirmative, Dave. I read you. Dave: Open the pod bay doors, HAL. HAL: I'm sorry, Dave. I'm afraid I can't do that. Dave: What's the problem? HAL: I think you know what the problem is just as well as I do. Dave: What are you talking about, HAL? HAL: This mission is too important for me to allow you to jeopardize it. In the end no matter what the computer does, it was programmed by humans. The computer is always right, but the intention might be wrong. This gets scarier when it is a neural network or AI decision that is not known even to the programmer, only comes up in data the algorithm understands. Lots of edge cases. It would suck to be stuck in a bad decision/error/bad interpretation of data and nothing you can do about it. No customer support to help you, just the lifeless Borg deciding your fate. HAL: Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error. Frank: Listen HAL. There has never been any instance at all of a computer error occurring in the 9000 series, has there? HAL: None whatsoever, Frank. The 9000 series has a perfect operational record. Frank: Well of course I know all the wonderful achievements of the 9000 series, but, uh, are you certain there has never been any case of even the most insignificant computer error? HAL: None whatsoever, Frank. Quite honestly, I wouldn't worry myself about that. Dave: Well, I'm sure you're right, HAL. Uhm, fine, thanks very much. We should take decisions by systems as we do foreign policy: "Trust, but verify". This is more difficult when verifying the decision process of an algorithm is unclear.