This is such an interesting topic and I really don't know where to stand on it. On one hand, this is an argument that systems such as laws, lending practices, e

This is such an interesting topic and I really don't know where to stand on it. On one hand, this is an argument that systems such as laws, lending practices, etc. should be based on repeatable and explainable rules. Rules will end up with some bias, but the bias can be dissected and argued about until we reach some consensus that the bias is acceptable/desiresble. In the case of lending, maybe we consider credit history a fair input metric but zip-code an unfair metric, for example. On the other hand, systems such as laws and lending, which are inherently about social interaction, do tend to have some unspecified human element. Judges interpret the law and apply it to specific cases, underwriters have latitude to make exceptions under certain vague circumstances, teachers may regrade a paper upon realizing they misspoke in a lecture. This is a feature, not a bug--if our social systems have no room for empathy then there is a big problem. So now that AI is "unexplainable", how is this worse than the unexplainability of human systems? You can ask a human why they took some action, but their explanation tends to be incomplete, wrong, a lie, or maybe they don't recall. I once was given the opportunity to lease a top-floor apartment in my building because I had a daily chat with the receptionist. All fair housing laws were followed, I just happened to be the first to know because I had such frequent interaction. If you were to ask the receptionist why I got the apartment she probably wouldn't say "zwkrt tells me bad jokes and we share pictures of our pets", but that probably is part of the reason. I think fundamentally we understand that humans share a way of life and a set of ineffable values, and we believe that computer-generated results do not share these values. Unfortunately I do not have a conclusion, just a formulation of a problem.