Let's say this is possible. How would we know that it (the AI) isn't doing a post-hoc rationalization, or just outright lying about its reasoning? In other wor

Let's say this is possible. How would we know that it (the AI) isn't doing a post-hoc rationalization, or just outright lying about its reasoning? In other words, why do we trust humans more than machines? In fact, why do we not think of humans as machines - just ones made out of different materials? Why do we have this bias that machines are and must-be deterministic, and since humans aren't, they must not be machines? Furthermore, since we know that these AI models are sometimes stochastic, why do we still insist that they be explainable; when humans exhibit the same kind of output, we don't insist upon their determinism...? I'm not certain that we can make these models - especially complex deep-learning CNNs and others like them - explainable, any more than an individual can tell you how his or her brain came up with the solution; most of the time, we employ post-hoc reasoning to explain our decisions, depending on how the output resolves. That - or we lie. Rarely do we say "I don't know" - because to do so is to admit a form of failure. Not admitting such is what helps religion continue, because when we don't know, we can ascribe the reason to another external force instead. If we would just be willing to say "I don't know - but let's try to find out" (insert XKCD here), we might be better off as a species. I don't think an AI model will be any different - or can be. If we insist on having an AI be able to deterministically and truthfully tell us exactly how it arrived at such a conclusion, we must be ready to accept that we should do the same with human reasoning as well. Anything less would be hypocritical at best.