Here is a paper on the topic of NN = Bayesian Posterior Probabilities: http://www.ee.iisc.ac.in/new/people/faculty/prasantg/downloads/NeuralNetworksPosteriors_

Here is a paper on the topic of NN = Bayesian Posterior Probabilities: http://www.ee.iisc.ac.in/new/people/faculty/prasantg/downloads/NeuralNetworksPosteriors_Lippmann1991.pdf http://www.ee.iisc.ac.in/new/people/faculty/prasantg/downloa... Re: comment on showing "top" variables from a model, I agree this could have utility. But I would add that the devil's in the details, and there are multiple ways to calculate importance values, each of which has its own nuances and pros/cons. For instance, how do you compare the importance of a categorical feature to a float feature? Do you one hot encode and then add their individual importances, take the average, or something else? Although sampling from the columns is meant to help deal with feature correlation, under what conditions is this effective and how do you know if your feature importances are safe? Moreover, how does this column sampling work in the context of one-hot-encoded categorical features? This is all a way of saying that while you can devise methods for coming up with metrics, and then assign them handy titles like "Feature Importance", the reality is that these things are pretty nuanced and limited, and upper level management might be fooling themselves by thinking they're "interpreting a model" if they don't recognize the limitations and nuances involved. Or to put it another way, to say "I better understand this model because you gave me a feature importance list and a partial dependence plot," is a dangerous over-simplification.