4 ms·
What frightens me more is how rarely I see talk about decision theory and hypothesis testing in the (deep) machine learning community, it is as if people consid
by lokimedes 9y ago
What frightens me more is how rarely I see talk about decision theory and hypothesis testing in the (deep) machine learning community, it is as if people consider the classification output as sufficient evidence of recognition, just quote that max(p(class)) rather than the significance of the class given its classifier score.
Am I missing something?
- nonbel 9y agoWhat do you think calculating the "significance" would add?
- lokimedes 9y agoMost dCNN classifiers I’ve seen used simply rely on softmax to provide a “probability” among the classes. But it really is a score that has been normalized across the classes. Having 0.7 of one class does not mean the same level of discrimination from the remaining classes as it does for another class with the same score. By only using the maximum scoring class you don’t account for what score value is sufficient to claim a significant discrimination between the maximum score and the alternatives.
- deleted 9y ago[deleted]
- nonbel 9y agoI see how the probability (softmax output) contains more info than simply the class with highest probability, but not what you want regarding "significant discrimination". Perhaps you want to weight different types of errors differently? Eg: https://github.com/Hezi-Resheff/paper-log-bilinear-loss https://github.com/Hezi-Resheff/paper-log-bilinear-loss
- deleted 9y ago[deleted]