The "UX design" terminology confused me. It seems that the author is advocating more for tools and methods that let us visualize & inspect how a given neural ne

The "UX design" terminology confused me. It seems that the author is advocating more for tools and methods that let us visualize & inspect how a given neural network comes to be based on its training data, and why it gives a particular output for a particular input. This ties back into the "neural networks are black boxes" argument that's been floating around for a while, no? Is there any concrete research in this field? The initiatives the author points to seem to be mainly about highlighting the problem, not addressing it. That being said, while I agree with it in principle, in practice what would compel companies to make their neural network based systems understandable and analyzable to users? There are plenty of non deep learning based systems in my life that I already don't have access to the internals of; for instance, it'd really be great if I could actually see the set of formulas that determine my credit score, for one example amongst many.