5 ms·
Ok, fair enough, the more sophistication you add, the less explainable it gets. That being said, you can look at magnitude and direction of various coefficient
by free_rms 6y ago
Ok, fair enough, the more sophistication you add, the less explainable it gets.
That being said, you can look at magnitude and direction of various coefficients and conclude certain obvious things. Even the case where reversing the correlation by picking some features instead of others says something -- those features point different ways but one has a higher magnitude so it 'wins'. They're ultimately pretty simple statistical observations about the training set.
A deep net, in comparison, is a totally unexplainable black box between the cross-linking and the sigmoid functions.
- qsort 6y agoI agree that there is an intuitive concept of "explainability", which is why I said that a formal model for it would be interesting to see. I also agree that "the average explainability of a deep net is lower than the average explainability of a linear model" (whatever that sentence means), my point is that it's extremely hard to pinpoint what explainability means, because the usual characterization of "you can fit the model in your head" is extremely subjective and brittle. I'm all for more research in this field. Heck, I'm all for more research in any field, but because our definition of "explainability" is so primitive and unrefined, arguing that we should build ethical or regulatory frameworks on top of it seems premature.
- glial 6y ago> you can look at magnitude and direction of various coefficients and conclude certain obvious things. Sometimes, but if you have correlated predictors, the 'explainability' can be distributed across coefficients of those predictors in unpredictable ways. Training a linear model twice with two different subsets of training data can give you very different coefficient weights.