4 ms·
Neural Networks Are Decision Trees
- stevenalowe 4y agoHere’s a list of 50,000 decision rules based on a sparse vector model of the universe = “interpretable”?
- version_five 4y agoThis equivalence shows that neural networks are indeed interpretable by design and makes the \textit{black-box} understanding obsolete Show me the decision tree for a big classifier trained on imagenet, and let's see if it's interoperable
- yarg 4y agoHow about the decision tree for even a simple recurrent NN.
- sirwhinesalot 4y agoThis is something I already suspected since you can encode piece-wise linear function-based neural networks (of a reasonable size) as mixed-integer problems (MIP). Since MIP is NP-complete, that means you could also translate the neural network more painfully into SAT, giving you all the interpretability you could possible want... Except I'm not too sure how "interpretable" the results really would be in most cases. What questions would you pose to an ML model recognizing images or audio?
- rrobukef 4y agoSecond order questions (i.e. get a really good machine): What is the minimal difference in input needed for class A to be recognized as class B? What the (an) input closest to all classes and what is the distance to each? Are there weights for this smaller NN that remain equivalent to the given NN?
- sirwhinesalot 4y agoThat makes a lot of sense, thanks!
- cuteboy19 4y agoWho says Decision Trees aren't black boxes? That was one of the first exercises taught after DTs: Decision Trees aren't always explainable. You need to prune them before getting anything useful
- hoseja 4y agoIsn't this a tautology to the tune of "a program running on a Turing machine is a program running on a Turing machine"?
- cuteboy19 4y agoRead the paper, it's quite short and sweet (though NN papers do have a lot of subscripts and superscipts) Basically yes, it was known that we can always do this. But apparently until now no one gave an algorithm to do so. I think it would probably be difficult to come up with a similar algo for sofmax