5 ms·
Neural nets are infamous for not doing this. Learning algorithms that produce decision trees are usually used in this situation.
by paulddraper 10y ago
Neural nets are infamous for not doing this.
Learning algorithms that produce decision trees are usually used in this situation.
- allenz 10y agoIt's theoretically possible for a neural net to do this; the network just needs to have the explanation as an output. I agree that decision trees would be more reliable and easier to train, but I'm not sure if hardcoding every feature is scalable.
- function_seven 10y agoHow do you know that the explanation jives with the other outputs, though? It seems like a turtles-all-the-way-down situation, because now I want to see how it was properly introspective of its own decision making. Also seems like it’s another magnitude of complexity in the neural net to have it not only train and learn on your inputs, but also train and learn on its own training and learning.
- snaky 10y ago>How do you know that the explanation jives with the other outputs, though? The third neural net would do the checking, obviously.
- allenz 10y agoNeural networks don't do anything as sophisticated as self-referential introspection. They just fit the outputs you train them with. The training data you provide would have to include the desired explanations. Consistency is enforced by the dataset, and also by the model. Both outputs would read from the same hidden layer--the one that encodes the desired transformation.
- randomsearch 10y ago> the network just needs to have the explanation as an output And how would you evaluate whether the explanation was correct or not?
- Houshalter 10y agoYou give it explanations as training data and it tries to predict them.
- nebabyte 10y agoWhich is why learning algorithms that produce decision trees are far smarter in the long run. Neural nets might eke out other benefits but there's a lot to be said about justifiable/accountable decisions.
- snaky 10y agoExcept the big (if not the main) part of modern economics is all about "don't care about long run - that's the only smart strategy".
- coldtea 10y agoExcept that's not so much a scientific theory but a justification for "quick bucks, everything else be damned".
- folli 10y agoCorrect me if I'm wrong, but the only type of decision tree that is comparable to a NN in terms of performance is an ensemble of decision trees, and these are equally hard to interpret as NNs.
- nilkn 10y agoThis might be a dumb question, but let's say that for whatever reason on a specific problem it's much easier to train a neural network that generalizes well than a decision tree. Why not train the network, then build an equivalent decision tree that just tries to reproduce the network's output? When building the tree from the network, overfitting would not be a concern. In fact, you'd want it to overfit. You could even say that it only needs to approximately reproduce the output with some tunable error threshold, which might give you leeway for finding more comprehensible and simpler trees.
- randomsearch 10y ago> This might be a dumb question, but let's say that for whatever reason on a specific problem it's much easier to train a neural network that generalizes well than a decision tree. Why not train the network, then build an equivalent decision tree that just tries to reproduce the network's output? When building the tree from the network, overfitting would not be a concern. In fact, you'd want it to overfit. You haven't fixed anything here. You've just encoded your training data in a neural net and then presented the same problem to the decision tree learner. Unless you're planning to transform your training data somehow?
- Houshalter 10y agoThis solves the problem of interpretability. You can't interpret the weights of a neural network, but you can easily follow along a decision tree and see if it's doing what you want. Actually that's somewhat less true for big decision trees. But the general point is that you can train interpretable models to mimic the output of uninterpretable black boxes. The biggest issue is that decision trees only work for data with fixed inputs and outputs. Recurrent NNs work on a time series and possibly even have attention mechanisms.
- jmmcd 10y agoNo, it doesn't make sense. The training data (inputs and NN-predicted outputs) that you're feeding into the DT is at best the same as the training data (inputs and desired outputs) you had originally.