There are ways to reverse engineer neural nets to try and understand the reasoning but AFAIK none of them are very good. For example, I went to a SAS/STAT cour

There are ways to reverse engineer neural nets to try and understand the reasoning but AFAIK none of them are very good. For example, I went to a SAS/STAT course a few years ago and one of the exercises was to train a simple neural network and then use it as input to generate a decision tree, that would (with pretty good accuracy, around 95% or so) explain the choices made by the neural network. I think the given scenario was that the neural network was used to decide who to send marketing emails to, and the head of marketing wanted to know why it was choosing certain customers. The problem with this was that it didn't offer insight into why decisions were made, it could only show what variable was being branched on at a given point in the tree. Also while it worked with this simple example, more complex NNs generated huge decision trees that were not usefull in practice.