Certainly humans can be unreliable. I guess my concern lies in the "black box" neural network design, by which, even if it mostly works great, it could also tro

Certainly humans can be unreliable. I guess my concern lies in the "black box" neural network design, by which, even if it mostly works great, it could also trot down some path that makes it randomly do something highly undesirable. Safeguards are fine if you know what to safeguard against. I suppose neither a human doctor nor an AI doctor should be "obeyed"; they are both there to assist the patient, not to control the patient.