4 ms·
The nice thing about using likelihood as the training criterion, you don't even need an extra category, you can just combine the outputs with basic decision the
by svantana 8y ago
The nice thing about using likelihood as the training criterion, you don't even need an extra category, you can just combine the outputs with basic decision theory, like so:
thres = wrong_cost / (wrong_cost+right_gain)
response = (max(out_probs) > thres) ? argmax(out_probs) : "I don't know"
While the bayesian methods can theoretically perform better for out-of-set inputs, it feels like an oversight to not include this simple method as a baseline. My gut feeling is that this would perform equally well, if not better.
- paraschopra 8y agoThis works if you get nicely balanced probabilities out of neural network. Softmax which is popular way to transform network outputs to probabilities pushes probabilities either to zero or one. There may be other ways to transform outputs to probabilities where your approach could work but I'm not aware of them. Also to reply to OP's comments, to have a class like 'I don't know' you need to have a dataset of things that don't look like your training dataset. And you'll always miss examples of things that don't belong to your input class. Bayesian solves it in a nice way by not requiring this infinite not-input training data.